Ethics and LLMs: Justice
In 2016, investigative journalists at ProPublica uncovered something disturbing about COMPAS, a widely-used algorithm that helps judges decide whether defendants should be released before trial. The system, which had been making these life-altering recommendations for over 15 years, was systematically biased against Black defendants.
The numbers were stark: Black defendants who never reoffended were twice as likely to be incorrectly labeled as high risk compared to white defendants. Meanwhile, white defendants who did reoffend were mistakenly labeled as low risk almost twice as often as Black defendants (ProPublica investigation, 2016).
Nearly a decade after these findings, COMPAS is still in use in the United States.
This example highlights two important points for mission-driven work:
AI is not an oracle. AI builds itself out of data that humans create. It reflects our society, even in terrible ways. It can recycle our bias and injustice, repackaging it as an “objective” computer output.
There is a need for algorithmic justice advocacy in the US and likely elsewhere. As AI is increasingly integrated into high-stakes decisions (like sentencing, policing, hiring and firing, for example), there will be a need for independent observation, clear guidance, and probably lots of lawyers.
The Hidden Threats to Equity
Beyond Prejudicial Outputs
In addition to visible bias like COMPAS, AI threatens equity in less visible ways. Consider how these systems can perpetuate harm:
Automation bias compounds the problem. Research spanning decades in aviation and medicine shows that humans tend to over-rely on automated systems. When a computer gives us an answer, we assume it's objective and free from human biases. This makes AI dangerous when the humans acting on outputs from a recommendation system or learning from an LLM’s ideas aren't thoroughly trained to question them.
AI can spread prejudices. Just as humans repeating prejudices to each other expands their negative impact, AI systems learn prejudice from training data and repeat it back to users. Combined with our confirmation bias and automation bias, we take this as neutral, authoritative confirmation of beliefs that support our prejudices.
Cultural Erasure Through Standardization
AI systems often impose dominant cultural frameworks as universal standards. An AI tutoring system that only recognizes academic communication styles may undervalue students who express knowledge in culturally specific ways. Mental health assessment tools that categorize conditions according to health insurance codes may obscure other schemas that are more relevant to specific communities.
Displacing Community Wisdom
Perhaps the subtlest threat comes when AI replaces human judgment and community knowledge with algorithmic decision-making. When organizations use AI to determine service eligibility or assess community needs, they may override community members' understanding of their own situations and priorities in favor of generic, contextless advice.
Turning AI Into a Tool for Justice
Despite these risks, the same technological capabilities that perpetuate inequity can advance justice when implemented with community leadership and explicit equity goals.
Pattern Detection for Accountability
AI's ability to process vast amounts of data can reveal discrimination patterns that are impossible for humans to detect manually. Machine learning systems can analyze thousands of court decisions, hiring practices, or service delivery records to identify disparities that might otherwise remain hidden.
Community organizations can use these findings to strengthen advocacy efforts. Pattern recognition can reveal whether environmental hazards concentrate in communities of color, whether school disciplinary practices disproportionately affect certain students, or whether lending practices systematically exclude particular neighborhoods.
Look for intersectional harm. Some discrimination patterns are harder for humans to spot. If women have slightly worse outcomes and Black workers have slightly worse outcomes, check whether Black women experience much worse outcomes. Machine learning can identify patterns across multiple variables that human analysts might miss: rural versus urban differences, impacts on people with invisible disabilities, or assumptions about family composition that exclude people who don’t fit the mold.
Proactive Resource Distribution
Assistance programs are notoriously underused. People don't know what they qualify for, applications are difficult, and stigma prevents participation. Instead of waiting for people to apply, AI systems can proactively identify communities and individuals who have been systematically excluded from opportunities.
GiveDirectly demonstrated this approach by using AI to analyze satellite images to identify areas of particularly high need, then nudging people in those communities to fill out a streamlined application. Machine learning can analyze multiple data sources to identify households at risk of housing instability or students who would benefit from additional academic support before crises occur.
Democratizing Sophisticated Analysis
AI tools can give smaller organizations analytical capabilities previously available only to well-resourced institutions. A neighborhood organization can analyze discriminatory lending patterns. A community health center can identify health disparities and target interventions more effectively.
With careful implementation and quality monitoring, smaller organizations can leverage these powerful tools with less risk than free experimentation and less cost than bringing in full-time expertise.
Strategic Implementation for Justice
Successfully using AI to advance rather than undermine justice requires fundamentally different approaches to technology development, deployment, and governance.
Essential Training for Staff
Many people working in mission-driven organizations (just as in the population at large) believe technology is neutral. They don't realize how feature design embeds bias or how AI recycles our biases back to us. Staff need to understand:
How their own biases affect technology use and AI outputs
How biases in training data create biased outputs
Methods to test for and mitigate algorithmic bias
The importance of model cards and documentation for vendor systems
Key Questions for Your Organization
Before implementing any AI system, ask:
Who benefits and who bears the risks? If errors occur, who experiences the consequences?
Whose knowledge and perspectives shaped this system? Were affected communities involved in design decisions?
How will we monitor for disparate impact? What metrics will reveal if the system creates different outcomes for different groups?
What recourse exists when the system makes mistakes? Can affected individuals challenge or appeal algorithmic decisions?
Are we replacing or supporting human and community judgment? Does this system enhance community capacity or substitute for it?
What equity considerations guide your organization's approach to AI? How are you ensuring community voices shape these decisions? Share your experiences and challenges as we work together toward more equitable AI implementation.
—
LLM Disclosure:
Tried asking both Claude Opus 4.1 and ChatGPT 5 to draft a blog post from a chapter of my book manuscript. Claude’s was much more in line with what I was hoping: GPT 5’s started with a generic statement about the importance of equity and some definitions of equity and justice that it must have inferred from the examples in the draft because they were centered around AI, not true definitions of the terms. I edited the Claude draft moderately.