Model risk management for AI, in proportion

The supervisory guidance on model risk management, issued as SR 11-7 by the Federal Reserve and OCC Bulletin 2011-12, predates today's AI tools by more than a decade. The FDIC adopted the same guidance for the institutions it supervises. Even so, the ideas in it are the ones examiners reach for when they ask how you govern AI. The good news is that the guidance talks about proportionality, and community institutions are not expected to build the program of a large bank.

Credit unions should ask their examiner how NCUA expects model governance to be applied, since the specifics differ. The substance below is still a sound way to organize the work.

What counts as a model

The guidance defines a model broadly, as a method that applies statistical, economic, financial or mathematical theory to turn inputs into quantitative estimates. A credit scorecard is a model. So is a fraud scoring tool, a prepayment estimate or an AI system that produces a score or a recommendation used in a decision. A tool that only summarizes a document for a person to read sits in a gray area, which is why you should decide deliberately and write the reasoning down.

The three parts of the framework

1. Development, implementation and use. Document what the tool is for, what data it uses, how it works at a level a non-specialist can follow, its known limits and how it was tested.
2. Validation. Someone independent of the builder checks that the tool does what is claimed and challenges the assumptions. At a small institution this can be an outside reviewer, internal audit or a qualified colleague who had no part in the build.
3. Governance and ongoing monitoring. Keep an inventory, assign an owner to each model, control changes, and track performance over time against thresholds you set in advance.

Make it proportionate

Not every tool needs the same effort. Tier your inventory by a few factors: how material the decision is, how complex the tool is, how much it affects customers or members, and how hard a mistake is to reverse. A document extraction tool with a reviewer checking every item sits in a lower tier than an automated underwriting score. Spend your validation effort in proportion.

What a model risk file contains

A short file is better than none, and a long file that nobody reads is worse. A workable version covers:

• Purpose and intended use, and what the tool must not be used for.
• Data sources and how they are controlled.
• A plain English description of the method.
• Known limitations and failure modes.
• Test results, including how it was tested and on what data.
• Performance thresholds that trigger a review.
• Named owners and a change log.
• A fallback if the tool is unavailable or fails.

What is different about AI

• Vendor opacity. Many tools are black boxes. You still need documentation, testing evidence and a commitment to notify you of material changes. See vendor due diligence.
• Drift and change. Performance can shift as data changes or as a vendor updates a model. Monitoring has to be scheduled, not occasional.
• Confident, wrong text. Generative tools can produce it. Constrain what they are allowed to do, keep a person reviewing output, and log prompts and results.
• Data exposure. Staff pasting member data into public tools is a real risk. A clear acceptable use policy and approved tools matter as much as any technical control.

The NIST AI Risk Management Framework is a useful, voluntary reference for vocabulary and structure. It does not replace supervisory expectations, but it helps when you are explaining your approach.

A practical first step

Build the inventory first. Include the tools that staff already use, formally or not, and the AI features that arrived inside software you already license. Classify each one by risk tier, name an owner, and note what documentation exists. You will almost certainly find more than you expected, and you will know where to focus.