Quaintitative

AI Supervision

AI oversight and accountability

The first thing you assess is whether one approach governs AI risk across the whole firm, and whether someone real is on the hook for it. The spine, and the accountability that holds it up. I supervised AI at the Monetary Authority of Singapore and wrote the AIRG, and the mistake I see most in AI oversight is blessing a structure instead of testing whether it does anything. Supervisors arrive here wanting to bless a structure. Firms want one blessed. Both are a trap.

What operating model should we expect of a firm - a central AI function, hub-and-spoke, or capability in every unit? And should we require a Chief AI Officer to own it?

A clean structure is comforting to approve. So I ask back:

If you approve a shape, what happens at the next firm, whose constraints are different? And if you name one accountable head, what happens when the levers they need sit in functions they do not run?

You do not supervise AI risk by approving an ideal shape or a tidy title. You supervise it by checking that one consistent approach runs through the firm, and that accountability for each AI outcome is real - held by someone who also holds the controls. The AIRG takes exactly this view: it does not impose an operating model, and it does not invent a new accountable persona. It requires consistency, and it slots AI into the three lines of defence you already supervise.

The spine: consistency, not a shape

The thing to look for first is not a diagram. It is whether the firm does the same few things the same way everywhere AI runs. Finds it the same way in every unit. Rates it on one scale. Holds it in one inventory. Attaches controls on the same basis. Reviews at a consistent intensity. I call that the spine, and it is what makes everything downstream mean anything.

Why it matters to you specifically: without it, the numbers a firm reports to you stop adding up. If one unit calls a model AI and another does not, their inventories do not reconcile, and the firm-wide picture you are handed is fiction. If a "high" on one desk is a "medium" on the next, you are being shown totals built from units that cannot be summed. A firm can have excellent controls in three places and no spine, and it is the no-spine that will hurt it, because what threatens a firm rarely sits inside one well-run unit. It sits in the gaps between them, where no single owner is looking.

So you do not ask the firm to prove it has centralised AI, or distributed it. You ask it to prove the method is consistent whichever way it is arranged - and that the arrangement would still hold if AI moved from a central team out to the desks, or back. The AIRG is explicit that the shape is the firm's choice and the consistency is not; your scrutiny should sit where the AIRG puts the requirement.

The board: challenge, appetite, competency

The board approves the approach and oversees it. Your test is not whether a board paper exists. It is whether the board did anything a rubber stamp would not.

Three marks tell you. First, challenge: the minutes should show a board that questioned and sometimes pushed back on AI proposals, not one that received and noted them. A board that has approved everything has overseen nothing. Second, appetite: material AI risk should sit inside the firm's risk appetite in terms that can actually be breached - qualitative statements paired with limits someone measures - not a sentence of aspiration. Third, competency: the board needs enough understanding of AI to challenge it. You are not checking for data scientists on the board; you are checking that the board's grasp is adequate to the complexity of what the firm has deployed, and that where it is thin, it is topped up with training or external expertise rather than left blank.

Senior management: run it, resource it, escalate it

The board owns the approach; senior management runs it. Here you look for three different things. That the board-approved framework has been translated into policies and procedures people actually work to, not left as principles. That the function is resourced in proportion to the AI estate - headcount and budget against the number, complexity, and risk of the systems deployed, because a two-person team overseeing three hundred models is a finding whatever the policy says. And that escalation works in the direction that matters: material AI risk and threshold breaches reaching the board in time to act, with evidence that issues get surfaced rather than quietly absorbed.

Resource proportionality is the one supervisors under-weight. A beautiful framework with nobody to run it is not oversight; it is intent. Count the people against the estate.

A board that has approved everything has overseen nothing.

The committee, and only where it is needed

The AIRG requires a dedicated cross-functional committee only where the firm's overall AI risk is material. That conditionality is itself something to check: how did the firm decide whether its overall exposure is material, and does that determination get revisited. A firm running material AI with no such committee, and no good account of why not, has a gap.

Where the committee exists, judge it the way you judge the board - by friction, not by terms of reference. Composition across the three lines and the key functions, seniority with real decision authority, a meeting cadence that matches the pace of deployment, and above all a record of decisions that bite: systems sent back, deployments made conditional, action items that actually closed. A committee whose minutes are a list of approvals and noted updates is oversight on paper, dressed formally.

Into the existing frameworks, not beside them

AI risk is not a new silo. The AIRG requires that the firm's existing risk frameworks - model, operational, technology, third-party, data, reputational - identify, assess, and address the risks AI brings, with clear ownership where AI risk spans several of them. So you check integration, not just the existence of an AI policy. Have the relevant risk policies actually been updated for AI, or does a shiny AI framework sit beside untouched old ones. Are AI definitions and risk ratings consistent across those frameworks, so the same system is not "high" in one and "medium" in another. And watch the exception log: frequent waivers from standard controls for AI deployments are a pattern, and the pattern usually means the controls are being routed around under deadline.

Accountability that holds the levers

Now the harder half: whether accountability is real. The appeal of a single AI chief is that it looks clean. The problem is that one person handed accountability for all of AI often cannot move the controls, because the controls live in functions they do not run - responsible for everything, able to change nothing. The financial sector solved this long ago, and AI does not need a new answer. It needs the three lines of defence.

The business owns the risk it takes and runs the first-line controls, but does not sign off on its own work. An independent second line challenges and validates - and for a material system, that validation is required, not discretionary, performed by people genuinely independent of the builders, with the standing to send a system back. The third line, audit, checks that both are working. On top, the AIRG expects a control function to be the firm-wide arbiter of what counts as AI and how material each use is, so identification and rating cannot be quietly set by whoever benefits from the answer.

So the question is not "is there a Chief AI Officer." It is whether, for each AI outcome, someone is accountable by name and actually holds the levers that control it - across the three lines, with independence where it counts. Test it with a live case: when this system last failed or was challenged, who was accountable, what could they actually change, and did the independent function have the authority to stop it. Accountability leaves the same marks oversight does. If no one has ever been able to say no, no one is really accountable.

For the supervisor

What to look for. One consistent approach across the whole firm (the spine), whatever the operating model - find it, rate it, control it, the same way everywhere, so the firm-wide numbers mean something. A board that challenges rather than notes, with material AI risk in an appetite that can be breached and competency adequate to what is deployed. Senior management that has turned the framework into practice, resourced it against the size of the estate, and escalates breaches in time. A cross-functional committee where overall AI risk is material, judged by the decisions that bite. AI folded into the existing risk frameworks, not sitting beside them. And accountability that is real - the three lines, with independent validation for material systems and a control function arbitrating identification and materiality - not a title that holds no levers.

Ask the firm:

  • Do you govern AI the same way across the whole firm, and would that still hold if you moved AI from a central team out to the desks?
  • Show me where the board challenged an AI proposal, and the AI limits in your risk appetite that someone could actually breach.
  • How many people run AI risk oversight, against how many systems - and how was that sized?
  • Where overall AI risk is material, show me a decision your AI committee made that changed or stopped something.
  • Have your model, operational, technology, and third-party risk policies been updated for AI - and how often do AI deployments get waivers from standard controls?
  • For a system that failed or was challenged, who was accountable by name, what could they actually change, and could the independent function have stopped it?

Work with me

I train regulators, supervisors, and public authorities on AI governance and risk management. See the courses and workshops, read more on AI risk management, or get in touch.