How Artificial Intelligence Can Deepen Racial and Economic Inequities

Proponents of expanding the use of artificial intelligence (AI) often point to its potential to stimulate economic growth — increased productivity at lower costs, a higher GDP per capita, and job creation have all been touted as possible benefits. The promise of an economic boost via machine learning is understandably seductive, and private and government actors are now regularly using AI in key areas of economic opportunity, including education, housing, employment, and credit, to name just a few. But as AI adoption is cast as a smart economic investment in the future, it’s important to pause and ask: Whose futures and whose wallets are we talking about?

There is ample evidence of the discriminatory harm that AI tools can cause to already marginalized groups. After all, AI is built by humans and deployed in systems and institutions that have been marked by entrenched discrimination — from the criminal legal system, to housing, to the workplace, to our financial systems. Bias is often baked into the outcomes the AI is asked to predict. Likewise, bias is in the data used to train the AI — data that is often discriminatory or unrepresentative for people of color, women, or other marginalized groups — and can rear its head throughout the AI’s design, development, implementation, and use. The tech industry’s lack of representation of people who understand and can work to address the potential harms of these technologies only exacerbates this problem.

There are numerous examples of the harms that AI can have. AI tools have perpetuated housing discrimination, such as in tenant selection and mortgage qualifications, as well as hiring and financial lending discrimination.

For example, AI systems used to evaluate potential tenants rely on court records and other datasets that have their own built-in biases that reflect systemic racism, sexism, and ableism, and are notoriously full of errors. People are regularly denied housing, despite their ability to pay rent, because tenant screening algorithms deem them ineligible or unworthy.

These algorithms use data such as eviction and criminal histories, which reflect long standing racial disparities in housing and the criminal legal system that are discriminatory towards marginalized communities. People of color seeking loans to purchase homes or refinance have been overcharged by millions thanks to AI tools used by lenders. And many employers now use AI-driven tools to interview and screen job seekers, many of which pose enormous risks for discrimination against people with disabilities and other protected groups. Rather than help eliminate discriminatory practices, AI has worsened them — hampering the economic security of marginalized groups that have long dealt with systemic discrimination.

That’s why today, the ACLU, the Leadership Conference on Civil and Human Rights, Upturn, and two dozen partner organizations are calling on the Biden administration to take concrete steps to bring civil rights and equity to the forefront of its AI and technology policies, and to actively work to address the systemic harms of these technologies. Just two weeks ago, many of the same groups also joined together in an in-depth response to a request for information by federal financial agencies on the use of AI, raising many of the same concerns. Many groups have also offered concrete policy recommendations to federal agencies on addressing technology’s role in discrimination in the domains of hiring, housing, and financial services.

Thus far, federal agencies that regulate industries using AI have not taken the steps necessary to ensure that AI systems are accountable to the people they impact or that they comply with civil rights laws. Federal legislative and regulatory efforts have not yet methodically undertaken the task of ensuring our civil rights laws protect vulnerable people from the harms exacerbated by today's technologies. In fact, while the Biden administration has made an overarching commitment to center racial equity throughout federal policymaking, the administration's emerging AI and technology priorities have lacked the necessary focus on equity for people of color and others who have been subject to discrimination and bias. The administration to date has overlooked necessary civil rights and civil liberties perspectives as AI and technology policies are being developed, which risks further perpetuating systemic racism and economic inequality.

The bottom line is that the administration and federal agencies must prioritize and address all the ways that AI and technology can exacerbate racial and other inequities and ensure that its policies and enforcement activities lead to more equitable outcomes. Decades of discrimination have left people of color and Black people in particular, women, and other marginalized groups at an economic disadvantage in the U.S. The Biden administration must work to reverse the trends that continue to this day, which must necessarily include an emphasis on how modern digital technologies perpetuate inequity. The economic and racial divide in our country will only deepen if the administration fails to do so.