Every city government says they want to be data-driven. But not that many cities do it right.

My experience as a practitioner on a civic innovation team has led me to believe that focusing on being “data-driven” is harmful for a city’s most vulnerable communities and neighbors unless it requires its data to be rigorous and intentional, and accounts for the power that civic employees have when we make even the most minuscule of decisions.

My previous job was on a team responsible for transitioning a mid-sized city in upstate New York toward “better data practices”, but what “better data practices” meant was interpreted in a multitude of ways across every department and even our own team members.

• Some of us thought that if the city was better at measuring its performance in numbers and metrics, we would enable departments to better prioritize employee hours, department funds and therefore public service delivery.

• Some believed that data was evidence, and leaned into every opportunity to run a randomized controlled trial, even if an intervention or desired outcome hadn’t been identified before said trial.

• Our leaders believed that data was something to be looked at. They expected that we deliver data visualizations in the form of dashboards, maps, and charts which the leaders then had a hard time interpreting and would sit without being updated for months at a time.

We were all right…and wrong.

Performance management is a powerful tool that every efficient and effective team needs, but it cannot fix unhealthy work-life expectations or a lack of psychological safety on a constituent-services team.

Randomized controlled trials are perfect for testing whether an incentive or intervention was effective at achieving what it set out to achieve, but they require being very precise in what the desired outcomes are upfront, and they cost significant taxpayer dollars.

Data visualizations of charts and maps tell stories in bite-sized chunks, but sometimes the same story could be told more clearly and quickly with a few constituent quotes (and for a time-strapped civic leader, a quote might be easier to interpret than a very thorough dashboard).

None of these interpretations of how to use data is harmful, but I observed firsthand how an unintentional and unguided implementation of data-use prevented important and urgent decisions from being made. Having the mayoral agenda push so strongly for “data-driven decision-making” without a clear definition or set of expectations attached to those words left civic employees to interpret them by ourselves, often to the detriment of our constituents.

A planning associate changes what policy decisions can be made when they decide to exclude an air quality variable in the dashboard that will be shown to the mayor about what transit services to prioritize. A communications team alters who hears about new city services when they spend their budget on a brochure program distributed at town halls instead of adverts at bus stops in lower-income neighborhoods.

Decisions that impact constituents aren’t just the flashy policing policies, or budget allocations for a new technology, or the creation of a new committee, they are the micro-decisions that go into how we interpret the story that our data tells. Without a clear directive or training in how to use that data or what the data is usable for, we risk misallocation of resources, decisions without grounding in context, and continued disinvestment in our most vulnerable communities.

What I encountered as data-driven decision-making lacked a rigor and cross-departmental implementation plan - it left out the lived experience of constituents. A narrative around data-driven decision making may be moving the needle on a government’s organizational performance, but it will only hold us back from healthy and thriving communities in the long-run unless we start owning up to the power each and every person in a government organization possesses in the decisions we make on a day-to day basis, and implement best practices that hold us accountable to making those decisions intentionally.