But in a zero-sum context, that means that the resources to do something about the information that is learned is siphoned off to the technology. Even more frustratingly, what data analysis does well is open up opportunities for experimentation and deeper exploration. But to treat data analysis as a savior to a broken system is woefully naive.ĭoing so obfuscates the financial incentives of those who are building these services, the deterministic rhetoric that they use to justify their implementation, the opacity that results from having non-technical actors try to understand technical jiu-jitsu, and the stark reality of how technology is used as a political bludgeoning tool. So of course there’s a desire to jump in and try to make sense of the data out there to make a difference in people’s lives. Any social scientist with a heart desperately wants to understand how to relieve inequality and create a more fair and equitable system. The opportunity to help people by understanding the complex interplay of contextual information is invigorating. The insights that large-scale data analysis can offer is inspiring. This doesn’t mean that these tools can’t be used responsibly. Worse, they’re helping usher in a diversion of liability because time and time again, those in powerful positions blame the algorithms.

Otherwise, all that these systems are doing is helping divert taxpayer money from direct services, to lining the pockets of for-profit entities under the illusion of helping people. Above all, they make visible the brutal pain points and root causes in existing systems that require an increase of services.

They cost money to train their users to use the data responsibly. They cost money to evolve with the domain that they’re designed to serve. Whether we’re talking about judicial decision making (e.g., “risk assessment scoring”) or modeling who is at risk for homelessness, algorithmic systems don’t simply cost money to implement.

When implemented ethically, these systems cost more. If you ever hear that implementing algorithmic decision-making tools to enable social services or other high stakes government decision-making will increase efficiency or reduce the cost to taxpayers, know that you’re being lied to.

Automating inequality