Five regulators’ AI questions read like an exam plan

The week

On March 29, 2021, five federal agencies, the Federal Reserve, the CFPB, the FDIC, the NCUA and the OCC, announced a request for information on financial institutions’ use of artificial intelligence, including machine learning. It asked how institutions used AI in fraud detection, customer service, credit underwriting and operations, and what challenges they faced. Its questions covered explainability, data quality and alternative data, overfitting, cybersecurity, dynamic updating of models, the use of AI by community institutions, oversight of third-party AI, and fair lending. The request appeared in the Federal Register on March 31, and the agencies later extended the comment period to July 1. It did not propose any rule.

A request for information can look like a regulatory non-event. Nothing became binding, and no AI rule followed. But the March 2021 RFI read like an examination plan, and that is how institutions should have read it. Its questions sort into four tests drawn from rules that already existed: explainability, data, change over time, and third parties. None was new. Together they told institutions that AI would be judged against model risk, fair lending and vendor management expectations already on the books, so there was no reason to wait for an AI rule.

Explainability: the adverse action problem

The RFI asked how institutions understood and explained the outputs of complex models. For credit, that is a compliance question with a specific legal hook. Under the Equal Credit Opportunity Act and Regulation B, a creditor that denies credit or offers worse terms must give the applicant the specific principal reasons. A model that cannot produce reasons the institution understands and can defend creates direct exposure.

In practice, institutions using machine learning in underwriting tend to pair the model with an explanation method that ranks the factors behind each decision, then map those factors to reason codes. The hard part is testing that the mapping is accurate. A reason code saying “insufficient credit history” must reflect what actually drove the score, not the nearest standard phrase.

Beyond credit, explainability matters for fraud and anti-money laundering models, where investigators need to know why an alert fired, and for any automated decision that affects a customer or member. The workable standard is that the institution can explain, in terms an applicant and an examiner would accept, why the model did what it did. Simplicity is one way to meet that standard, but not the only one.

Data: quality, lineage and proxies

The second set of questions concerned the data feeding models. Machine learning can find patterns traditional methods miss, including patterns that act as proxies for protected characteristics. The RFI asked how institutions managed data quality and alternative data, such as cash-flow or rent payment data, and how they guarded against discrimination.

For most institutions, the binding constraint is data governance. They need to know where training data came from, what population it represents, and whether that matches the population the model will score. A model trained on approved applicants from one region, then used across a new market, can perform well in validation and badly in production. Fair lending testing then needs its own data, because institutions generally cannot collect race or ethnicity for most non-mortgage credit and must estimate it to test outcomes.

Change over time: models that keep learning

The RFI specifically raised dynamic updating, meaning models that retrain or adapt on new data. Traditional model risk management, set out in the Federal Reserve’s SR 11-7 and the OCC’s parallel 2011 bulletin, assumes a model is developed, validated, approved and then monitored. A model that changes itself challenges that sequence.

Most institutions converged on treating retraining as a controlled change. That means defined triggers, such as a drop in a performance measure or a shift in input distributions, thresholds that must be met before a retrained version goes live, and documented approval. Validation then shifts from approving a fixed artifact to approving the retraining procedure and its guardrails. The approach fits the existing framework. It also needs monitoring infrastructure many smaller institutions did not have.

The timing of the RFI made this concrete. Models trained on data from before 2020 met a world of stimulus payments, payment forbearance and sudden shifts in spending. Fraud and credit models that had been stable for years began to drift, and some institutions found they had no routine way to detect it beyond complaints from the business. Drift monitoring, with named owners and agreed responses, turned out to matter for static models as much as for self-updating ones.

Third parties: the vendor’s model is your model

The final test was oversight of AI supplied by vendors. Community banks and credit unions rarely build models. They buy fraud scoring, credit decisioning and chatbot services, or switch on AI features inside platforms they already use. The long-standing supervisory position is that using a vendor does not transfer accountability.

That creates a practical tension. Vendors often treat model details as proprietary, while institutions need enough transparency to validate, explain and monitor. Contracts resolve it, through rights to model documentation, independent validation reports, performance and fairness testing results, and advance notice of material changes. The RFI’s separate question on community institutions pointed here. Smaller institutions face the same expectations with far fewer specialists, so most of the four tests land on vendor due diligence: asking the right questions before purchase and tracking outcomes such as approval rates, alert volumes and complaint themes after go-live. A short annual attestation from each AI vendor, covering changes made, testing performed and incidents, gives a small compliance team something concrete to review.

With hindsight

Looking back from 2026, the notable fact about the RFI is that no AI-specific banking rule followed. Supervisors kept applying model risk, fair lending, consumer protection and third-party guidance, and the agencies’ 2023 interagency guidance on third-party relationships reinforced the vendor test. The CFPB said in 2022 and 2023 that creditors using complex algorithms must still give specific and accurate adverse action reasons. Generative AI arrived in late 2022 and widened the use cases to customer-facing chat, document drafting and code generation, with new risks such as fabricated outputs and data leakage. The four tests held up as a structure. What changed is scale: institutions now run many more models, many embedded in vendor products. Federal policy toward AI regulation and consumer enforcement also shifted after January 2025, so institutions should confirm current supervisory expectations with counsel.