AI model validation in finance

Overview

The need for validation is common across frameworks. Much of it traces back to a common origin: SR 11-7, the US model risk guidance from 2011. A model is validated three ways - you check it is conceptually sound, you test it before you use it, and someone independent challenges it. That last part is effective challenge: critical review by objective, informed people with enough competence and standing to push back and force a change.

Every broad AI framework restates this. The MAS AIRG asks for development standards plus independent validation, or peer review, scaled to how material the model is. SS1/23 makes independent validation its own principle. OSFI E-23 wants reviews that confirm a model is fit for purpose. The EU AI Act requires testing and a conformity assessment before a high-risk system reaches the market. NIST and ISO 42001 say the same for any sector.

Two things change for AI.

• Validation scales to materiality. The AIRG sets the depth of validation by how material the model is, and the risk-based tiering in SS1/23 and OSFI E-23 does the same. A high-impact credit or capital model gets full independent validation. A low-risk tool may need only peer review. The more complex and opaque the model, the deeper and more independent the challenge.
• The techniques stretch. More weight falls on testing, robustness, and explainability. Validation has to confirm the model still performs on new data, guard against overfitting, and check the outputs can be reproduced in production. Generative AI moves to outcome-based and human evaluation, benchmarking, and red-teaming, because a generative model has no single right answer to back-test against the way a credit model does.

Where they differ is how hard they push, not what they ask for. The EU runs a formal conformity assessment. The UK and US apply principles to every model. All of them want rigorous, independent validation before the model goes live.

What each framework says

MAS AIRG - validation is a lifecycle control, calibrated to risk materiality.

• Algorithm and feature choices must be justified against simpler alternatives.
• Testing is proportionate and broad - out-of-sample, stability, sub-population, stress and adversarial methods, with overfitting guarded against.
• High-risk AI gets independent pre-deployment validation of conceptual soundness, data, fairness, explainability and limitations, plus a technology and cybersecurity review.
• Generative AI is tested harder for hallucination, data leakage and adversarial attacks.

Guidelines on AI Risk Management (MAS, 2025)

United States, SR 11-7 - three-element validation under effective challenge. Conceptual soundness, ongoing monitoring and outcomes analysis, reviewed by objective, informed parties able to probe a model and force change. The 2026 revision carries this forward and puts generative and agentic AI out of scope. SR 11-7 / OCC 2011-12 (Federal Reserve / OCC)

United Kingdom, PRA SS1/23 - independent review of every model. The technique must be conceptually sound and supported by research or accepted practice, and risk-based tiering prioritises validation effort. The principles apply to all models regardless of technique, so they reach AI and machine learning models, which the companion DP5/22 confirms. SS1/23: Model risk management principles for banks (Bank of England / PRA)

United Kingdom, PRA / FCA DP5/22 - AI's complexity may need more supervisory focus on testing, validation and explainability than traditional models require. DP5/22: Artificial Intelligence and Machine Learning (Bank of England)

Canada, OSFI E-23 - reviews scaled to the risk rating. Model reviews confirm a model is properly specified, working as intended and fit for purpose, triggered by new models, changes, performance breaches or scheduled review, and the model must be shown to operate as expected in the production environment. Guideline E-23: Model Risk Management (OSFI)

European Union, AI Act - test and conformity-assess before market. High-risk systems run an iterative lifecycle risk process, must meet accuracy, robustness and cybersecurity standards, and must pass a conformity assessment before deployment. Regulation (EU) 2024/1689, the Artificial Intelligence Act

ECB, internal-models regime - an independent validation function, and machine learning raises the bar. Validation assures model quality including back-testing; the 2025 guide adds a machine-learning chapter treating ML as a driver of higher complexity, materiality and validation expectations. Guide to internal models (ECB Banking Supervision, 2025)

NIST AI RMF - the Measure function: test, evaluate, verify, validate. Metrics, validity, reliability and robustness, measured independently where possible and documented through formal, transparent reporting. AI Risk Management Framework 1.0 (NIST)

ISO/IEC 42001 - a documented pre-deployment gate. Verification and validation across the AI lifecycle, backed by internal audit of the management system. ISO/IEC 42001:2023, AI management system

MAS Veritas (FEAT) - the principles turned into verifiable assessment steps. A repeatable way to test and document AI and data-analytics systems before and after deployment. Veritas Document 3: FEAT Principles Assessment Methodology (MAS)

Hong Kong, HKMA - rigorous validation and testing before production use, to confirm the accuracy and appropriateness of AI models. High-level Principles on Artificial Intelligence (HKMA)

Hong Kong, SFC - validate and monitor generative-AI models. Adequate validation and performance assessment, with ongoing review, applied in a risk-based way to the use case. Circular on the Use of Generative AI Language Models (SFC)

IOSCO - testing and monitoring before deployment, so systems behave as expected under varied conditions, with senior management accountable for the lifecycle. Artificial Intelligence in Capital Markets (IOSCO, 2025)

IAIS - regular, documented performance assessment that accounts for known model and data limitations. Application Paper on the Supervision of Artificial Intelligence (IAIS)

AIR and Google Cloud - outcome-based and human evaluation for generative AI. Grounding, benchmarks and human evaluation with continuous monitoring, robust testing and human-in-the-loop, because conventional accuracy statistics alone do not capture generative risks like hallucination. Generative AI Risk Management in Financial Institutions (AIR and Google Cloud)

FAQ

What do the frameworks share on model validation, and where does it come from? SR 11-7's three core elements - conceptual soundness, ongoing monitoring, and outcomes analysis - plus effective challenge by independent, competent, authorised reviewers. Later frameworks restate and extend it rather than replace it, so a firm with a mature model risk management function already has most of what AI validation needs.

Does validation have to be independent, or is peer review enough? It depends on materiality. SS1/23 requires independent review of all models; OSFI E-23 allows internal reviewers or objective third parties with depth scaled to the risk rating; and the MAS AIRG accepts independent validation or peer review calibrated to risk materiality. Higher-risk AI attracts genuinely independent challenge; lower-risk tools may be peer-reviewed.

How does the EU AI Act's approach differ from SR 11-7's? The EU AI Act imposes a formal, documented conformity assessment and binding accuracy, robustness and cybersecurity requirements for high-risk systems before they reach the market. SR 11-7 and SS1/23 instead apply validation principles to all models, whatever the technique, through supervisory expectation. The EU route is prescriptive and gated; the US and UK route is principles-based and continuous.

Why does generative AI need different testing? A generative model has no single right answer to back-test against the way a credit model does. The guidance shifts to outcome-based and human evaluation, grounding and retrieval quality, benchmark suites, guardrail and red-team testing, and human-in-the-loop review, because the risks - hallucination, prompt injection, data leakage - are not captured by conventional accuracy statistics.

Does using machine learning automatically raise validation expectations? In the ECB's internal-models regime, yes: it treats machine learning as a driver of complexity and therefore materiality, which brings higher expectations towards reporting and internal validation. Other frameworks reach the same place through risk-based tiering: more complex, more opaque models earn deeper and more independent validation.

Read the sources

Every framework quoted above is in the full AI risk management in finance resource list. For the Singapore picture, see the AIRG explained and the AIRG compared to the EU AI Act, NIST and ISO 42001.

Work with me

I train banks, insurers, and supervisors on turning model validation and AI risk management into a working system, grounded in the AIRG and the frameworks above. See the courses and workshops, or get in touch.