How OMB’s first AI memo reached AI systems agencies already ran

The week

On October 30, 2023, President Biden signed Executive Order 14110 on the safe, secure and trustworthy development and use of artificial intelligence. One of the longest executive orders in years, it directed work across many agencies. That included reporting by developers of the most powerful models under the Defense Production Act, NIST work on standards and red-teaming guidance, and measures on AI in critical infrastructure, health care, housing and employment. It also addressed the government’s own use of AI. On November 1, the Office of Management and Budget released a draft memorandum on agency AI governance and risk management for public comment. It proposed chief AI officers, governance boards, and minimum practices for uses that affect rights or safety, with a deadline after which non-compliant uses would have to stop. The same week, the United Kingdom hosted an international AI safety summit at Bletchley Park.

Most coverage of the order dwelt on frontier models and national security. For agency program directors, the consequential document was the OMB draft released two days later, and its most consequential feature was easy to miss. Its reach extended past new AI projects. It defined categories of AI presumed to affect rights or safety, such as uses that inform eligibility for benefits, flag people for investigation, or control physical systems. It then said that any such use, including ones already running, must meet minimum practices by a set date or be discontinued. The draft proposed August 2024. Agencies that read the memo closely saw a retrofit program with an off switch. We look at that program in three parts, which follow the order of the work: finding the systems in scope, retrofitting the practices, and preparing for the stop.

Finding the systems in scope

Agencies had been required to inventory AI use cases under an earlier executive order, and compliance had been uneven. The new draft raised the stakes, because being in scope now carried a deadline. The difficulty was that many presumed-impacting uses were not labeled as AI. A fraud-scoring module in a benefits system, a risk model in a grants management platform, a prioritization score in an inspection scheduling tool, or identity verification with facial matching might all qualify. Many came from vendors, embedded in products the agency bought for other reasons.

Finding them meant reading systems, not surveys. An agency could start from the presumption categories and work backward: list every decision process that affects eligibility, enforcement, safety or access to services, then ask which systems supply a score, ranking or recommendation to that decision. Contracting officers could query vendors on active contracts about model-based features, with a deadline for written answers. Program offices, not the CIO alone, had to confirm the results, because they know which outputs staff actually rely on.

Retrofitting the practices

The draft’s minimum practices for rights- and safety-impacting uses began with an impact assessment, real-world testing and independent evaluation. They continued with ongoing monitoring, adequate human review, notice to affected people, and ways to appeal or opt out. Applied to a new project, these are design requirements. Applied to a system that has run for years, each becomes a retrofit with its own obstacles.

An impact assessment needs documentation of training data and model behavior that a vendor may regard as proprietary. Real-world testing needs outcome data that the program may never have collected, such as whether flagged cases turned out to be fraud. Notice and appeal require changes to letters, call center scripts and hearing procedures, and in some programs to regulations. Monitoring requires someone to own a performance report each quarter.

Human review deserves particular care, because it is the practice most easily satisfied on paper. A caseworker who approves nine hundred flagged cases a week is not reviewing the model. A meaningful retrofit measures how often staff disagree with the score, gives them the information and time to do so, and records the reasons. If the override rate is near zero, the agency has learned something important about how the system really works, whatever the procedure manual says.

Two moves made the retrofit tractable. The first was triage: rank in-scope systems by how directly their output drives decisions about people, and start with those where staff rarely override the score. The second was contract modification: add clauses requiring vendors to supply model documentation, testing access and notice of model changes, using the modification process rather than waiting for re-competition. Both needed budget, and the deadline fell inside a fiscal year whose appropriations had not anticipated it.

Preparing for the stop

The draft’s off switch was its sharpest feature. If an agency could not bring a use into compliance by the deadline, it would have to stop using it, unless it obtained a waiver through the chief AI officer for a documented reason. For a program that has woven a model into daily casework, stopping is not trivial. Caseloads rise when a triage score disappears. Fraud controls weaken. Processing times lengthen.

A responsible plan therefore included a fallback for every in-scope system: the manual or rules-based process that would replace the model, the staffing it would need, and the service effects residents would see. Writing that fallback had a second benefit. It forced program leaders to state what the model actually contributed, which is the core of any honest impact assessment. Where the fallback looked unacceptable, the agency had a documented basis for prioritizing that system’s retrofit or for a reasoned waiver request, rather than an argument made at the deadline.

The chief AI officer role, which the same draft created, was where these decisions met. An officer with authority over the inventory, the retrofit queue and waiver recommendations could make the trade-offs visible to agency leadership. One without that authority would end up reporting the deadline rather than managing it.

With hindsight

OMB finalized the guidance as memorandum M-24-10 in March 2024 and moved the deadline for minimum practices on rights- and safety-impacting uses to December 1, 2024. Agencies published expanded use-case inventories that December, and some issued extensions or waivers for specific uses. President Trump revoked Executive Order 14110 on January 20, 2025. In April 2025, OMB replaced the 2024 memoranda with M-25-21 and a companion memo on acquisition. The new guidance put more weight on adoption, but it kept chief AI officers, inventories and a category of high-impact AI. It again told agencies to document minimum practices for those uses within about a year or safely discontinue non-compliant functionality, subject to waivers. A later article in this series covers the 2025 transition. The comply-or-stop structure, and the retrofit work on systems already in production, outlasted the order that started it.