The Next Front in NYC’s Fight Against Discrimination: Algorithms

New task force will look at bias in the city's "smart" systems. 

(AP Photo/Mark Lennihan)

This is your first of three free stories this month. Become a free or sustaining member to read unlimited articles, webinars and ebooks.

Become A Member

As more cities embrace big data, algorithms play a growing role in the day-to-day activities of civic life, determining things like which school a child will attend or whether a person is likely to be allowed out of jail with bail. But all data comes from human sources, and humans are, of course, biased. A new bill passed Monday by New York City Council aims to correct some of those innate biases and make sure the city’s computerized systems are, in fact, fair.

The bill would establish a task force to monitor the algorithms used by municipal agencies, Tech Crunch reports. It currently awaits the mayor’s signature, and is being supported by the New York ACLU.

“Flawed code can … further entrench systemic inequalities,” the organization wrote in a recent release. “The algorithms used in facial recognition technology, for example, have been shown to be less accurate on black people, women and juveniles, putting innocent people at risk of being labeled crime suspects.”

A 2016 ProPublica study found that software used across the country to make bail and sentencing decisions were biased against black people. The tools are designed to determine the likelihood of future criminal activity, but the predictions were correct only 20 percent of the time.

The task force will write a report that explores whether given systems disproportionately impact certain groups, like the elderly, immigrants or the disabled. It will also examine how people should be informed as to whether they’re being assessed algorithmically and how those systems should be documented and archived, among other things. It would need to be formed within three months of the bill’s signing, and according to the bill include “persons with expertise in the areas of fairness, accountability and transparency relating to automated decision systems and persons affiliated with charitable corporations that represent persons in the city affected by agency automated decision systems.”

As Tech Crunch’s Devin Coldewey puts it, “this wouldn’t just be a bunch of machine learning experts and a couple of lawyers.”

The bill sets a welcome precedent for other cities, where leaders often struggle with how to regulate their data systems. It could also dovetail with efforts like Data for Black Lives, which seeks to build bridges between chief data officers and the black community regarding issues like displacement and economic opportunity.

Like what you’re reading? Get a browser notification whenever we post a new story. You’re signed-up for browser notifications of new stories. No longer want to be notified? Unsubscribe.

Rachel Dovey is an award-winning freelance writer and former USC Annenberg fellow living at the northern tip of California’s Bay Area. She writes about infrastructure, water and climate change and has been published by Bust, Wired, Paste, SF Weekly, the East Bay Express and the North Bay Bohemian

Follow Rachel .(JavaScript must be enabled to view this email address)

Tags: new york cityracism

Next City App Never Miss A StoryDownload our app ×

You've reached your monthly limit of three free stories.

This is not a paywall. Become a free or sustaining member to continue reading.

  • Read unlimited stories each month
  • Our email newsletter
  • Webinars and ebooks in one click
  • Our Solutions of the Year magazine
  • Support solutions journalism and preserve access to all readers who work to liberate cities

Join 1107 other sustainers such as:

  • John in Dayton, OH at $120/Year
  • Andrea at $100/Year
  • Caryn at $120/Year

Already a member? Log in here. U.S. donations are tax-deductible minus the value of thank-you gifts. Questions? Learn more about our membership options.

or pay by credit card:

All members are automatically signed-up to our email newsletter. You can unsubscribe with one-click at any time.

  • Donate $20 or $5/Month

    20th Anniversary Solutions of the Year magazine

has donated ! Thank you 🎉