When vendors set facial recognition policy before agencies did
IBM, Amazon and Microsoft stepped back from selling facial recognition to police within days in June 2020. Agencies discovered that suppliers were setting AI policy for them, because few had their own inventory, use rules or contract terms.
Looking Back revisits one week at a time, from January 2020 to today, and asks what it meant for the banks, health systems and public agencies we serve. The series was written in 2026, with the benefit of hindsight, and each article says which week it looks back on.
The week
On June 8, 2020, IBM told members of Congress in a letter that it would no longer offer general-purpose facial recognition or analysis software, citing concerns about mass surveillance and racial profiling. On June 10, Amazon announced a one-year moratorium on police use of its Rekognition facial recognition service. On June 11, Microsoft said it would not sell facial recognition to US police departments until there was a national law grounded in human rights. The announcements came during nationwide protests following the killing of George Floyd in Minneapolis on May 25. In December 2019, a NIST evaluation had found demographic differences in the accuracy of many facial recognition algorithms, with large variation between algorithms.
The June 2020 announcements were made by vendors, under public pressure, in a matter of days. Police departments and other agencies that used or planned to use facial recognition found that a supply decision was now a policy decision. Many had no formal policy of their own to set against it. Three controls were missing: an inventory of which tools were in use, rules for how they could be used and on what evidence of accuracy, and contract terms that made those rules enforceable. The same three apply to any AI system that supports decisions about people.
An inventory of tools in use
Facial recognition entered many agencies by routes that bypassed central governance. Free trials were offered directly to investigators. Features arrived embedded in other products, such as evidence management or video analytics. Detectives searched databases run by other jurisdictions, and some used services marketed to individual officers. Some agencies could not say whether their staff used facial recognition, which systems were involved, or which image collections were searched.
An inventory fixes that. For each tool it records the vendor, the algorithm and version, the image sources searched, the units and people with access, the purposes approved and the legal authority relied on. It includes capabilities embedded in other products and services reached through partners, such as a state or regional system. The same algorithm can be appropriate for one purpose and not another, so purpose belongs in the record. An inventory is also the precondition for answering legislators, auditors, defense counsel and the public, all of whom began asking in 2020.
Keeping the inventory current needs a rule with teeth. No algorithmic tool may be used on agency cases unless it is registered and covered by an agreement. Trials and embedded features count. Without that rule, the inventory decays as soon as it is published.
Rules for use and evidence of accuracy
Use policies answer practical questions. For which offenses or purposes is a search allowed? Can a match be the sole basis for action? Who reviews candidate matches, with what training, and how is the review documented? When is the public, or a defendant, told that facial recognition was used?
Accuracy evidence belongs in the policy. The NIST evaluation showed that accuracy varied by algorithm and by demographic group, and that the best algorithms performed far better than others. Vendor claims could not substitute for evidence about the specific algorithm, on the image quality the agency actually used, for the specific purpose. Low-quality probe images, such as frames from surveillance video, produce much less reliable results than the test conditions vendors cite.
The cases that later drew most attention involved people wrongly identified and arrested, where a facial recognition result had been treated as more than an investigative lead. A rule requiring independent corroboration before any action against a person addresses exactly that failure. So does training that teaches investigators that a ranked list of candidates is not an identification. Auditing a sample of searches each quarter shows whether the rules are followed in practice.
Governance of the rules matters as much as their content. Policies written by the unit that uses the tool tend to favor convenience. Agencies that involved legal counsel, a privacy officer and, in some jurisdictions, a civilian oversight body produced rules that held up better under public scrutiny. Publishing the policy, along with annual counts of searches and of cases where a match led to an arrest, gave legislators and residents a basis for trust. Those disclosures also forced agencies to collect the data needed to know whether the tool was helping at all.
Contract terms that make the rules enforceable
Procurement is where policy becomes enforceable. Contracts can require disclosure of the algorithm and its independent test results, logging of every search, audit rights, limits on the vendor’s use of agency data, and notice before material changes to the model. Without such terms, an agency cannot verify that a tool meets its policy. The vendor can also change or withdraw the product without warning, as the June announcements showed.
Contracts need exit terms as well. When a supplier withdraws, the agency needs its data back and its search logs preserved for oversight and litigation. It also needs a plan for investigative work that depended on the tool. Few contracts signed before 2020 contemplated a vendor leaving the market for policy reasons. Retention of logs matters most, because past searches may be challenged in court years later.
Facial recognition was the most visible case, but the three controls apply to any AI that supports decisions about people. Benefits fraud scoring, identity verification and eligibility screening carry similar risks of error falling unevenly on different groups. Agencies that build the inventory, rules and contract terms once can scale them to each use by risk. Waiting for a vendor announcement or a statute leaves the policy to someone else.
With hindsight
Amazon extended its moratorium on police use of Rekognition indefinitely in 2021. Several cities and states passed their own restrictions on government use of facial recognition. Other agencies kept using services from smaller vendors, including a tool built on images scraped from the internet that drew regulatory action in several countries. Wrongful arrests based on facial recognition matches, reported from 2020 onward, led to lawsuits and settlements that required changes to police practices.
Federal AI governance developed in stages. OMB issued government-wide requirements for agency AI use in March 2024, with added safeguards for uses affecting rights and safety. It replaced them with revised memoranda in April 2025 that kept inventories and risk management for high-impact uses. Under both versions, the inventory, use rules and contract terms agencies lacked in 2020 are the core obligations.