Victory for Principle, Dead End for Victims: The Algorithmic Accountability Gap

One-line summary

Regulators can ban biased AI systems, but the people harmed rarely get compensation or their day in court.

U.S. regulators increasingly use consumer protection law to police algorithmic harm, but these enforcement actions focus on stopping future wrongdoing rather than compensating individual victims. The FTC's Rite Aid settlement exemplifies this pattern: a landmark ban on facial recognition with no damages for those wrongly flagged as shoplifters. This creates a gap where algorithmic systems cause real harm to individuals who have no practical path to legal recourse.

In December 2023 the Federal Trade Commission did something that no federal agency had done before. It banned a large retailer—the pharmacy chain Rite Aid—from using facial recognition technology in its stores for five years, and it ordered the company to delete every algorithm and image it had collected through that system. The agency had concluded that Rite Aid’s AI surveillance falsely flagged thousands of customers, disproportionately women and people of color, as likely shoplifters or otherwise suspicious, leading to embarrassing public confrontations and, in some cases, police involvement. But if you were one of the people wrongly stopped, searched, or humiliated by a store employee acting on a false AI match, the FTC’s order did not require Rite Aid to pay you a dollar. The entire enforcement action—groundbreaking in its use of consumer-protection authority to govern biometric AI—jumped past the injured individuals and landed squarely on stopping future harm. It was a victory for the abstract principle that companies must not deploy biased surveillance, and a dead end for anyone who wanted a day in court or a check for damages. That lopsided result captures what is quietly becoming the dominant pattern in the United States: when algorithmic systems fine, fire, or exclude workers and consumers, the sharpest legal responses are coming not from labor law or even from private lawsuits, but from state attorneys general and federal regulators using broad consumer-protection statutes. Enforcement is accelerating. The ability of individuals to be made whole is not. A common assumption among people watching these developments is that robust government enforcement will eventually make individual lawsuits unnecessary—that if the FTC or a state AG can levy a big fine or impose a structural remedy, the problem is solved and workers are protected by proxy. That assumption is worth challenging directly. The Rite Aid settlement is a powerful rebuttal. The agency acted under its authority to police unfair or deceptive trade practices, not under an employment or civil-rights framework. The theory was that deploying a biased facial-recognition system without adequate safeguards was itself an unfair practice that deceived consumers about their safety in Rite Aid stores. That argument turned out to be legally potent. It required no proof of intent to discriminate, no showing of malicious motive, and nothing resembling the kind of causation chain an individual plaintiff must build to survive a motion to dismiss. The government could act precisely because it did not have to prove that any specific person was harmed in a legally compensable way. The result was a forward-looking ban, not compensation for specific wrongs. Consumer-protection law has become the unexpected weapon of choice against algorithmic harm. The iTutorGroup case is instructive: in 2023 the FTC secured a consent decree requiring the online tutoring platform to pay $365,000 to more than 2,000 applicants who were automatically rejected by hiring software that screened out older workers. In that instance, money did reach the individuals who were harmed, but only because the agency’s complaint alleged a clear, quantifiable injury—lost job opportunities—and because the settlement was structured, unusually, to distribute funds. That is the exception. For most workers facing an algorithmic penalty—a misclassification that docks pay, a scheduling tool that cuts shifts, a performance algorithm that flags them for termination—the private lawsuit route remains deeply treacherous. The boxes they must tick are formidable: identify a specific employer action, link it to a discriminatory outcome (often across a class), overcome the evidentiary opacity of a proprietary system, and then find a legal theory that makes the algorithm’s developer or vendor a proper defendant. The class-action litigation in Mobley v. Workday, Inc., which accuses the HR platform of facilitating discriminatory screening through its algorithms, is still grinding through preliminary motions years after it was filed. That pace is out of sync with the speed at which automated decisions take money out of a paycheck. So the justice gap widens. Regulators can move faster because they are litigating on behalf of the public interest, not to remedy individual harm. That strategic advantage is also the source of the limitation. Their remedies—deletion orders, bans, compliance reporting—stop systems. They rarely compensate people. The tension is unlikely to resolve itself, because the doctrines that empower public enforcement and the doctrines that constrain private lawsuits are part of the same legal architecture. Changing one would require changing the other, and that demands a legislative appetite that does not yet exist. For advocates, the shift from “you can’t sue” to “enforcement is outpacing your lawsuit” reframes the problem in a useful way. Public actions can freeze the most dangerous uses of algorithmic management and establish deterrents. But they are not a substitute for what a worker might get in a court that could award back pay and damages. The question is whether we are willing to tolerate a system in which the state can stop the machine but cannot restore what the machine took. The Rite Aid settlement forces that question to the surface. It has not yet been answered.

Victory for Principle, Dead End for Victims: The Algorithmic Accountability Gap · Soulstrix