The five questions a supervisor should ask about AI

Here is where I usually start: five questions a supervisor can put to a firm about its AI risk management. I did not invent them. They are the oversight duties from the MAS AI risk management guidelines (AIRG), read as plain questions you can ask from the outside. I supervised AI at the Monetary Authority of Singapore and wrote the AIRG, and these are the five I come back to. The areas are the MAS'. The interpretations here are wholly mine, so not regulatory guidance. Blame me if you disagree. For each one, the common question I get, and the one I would put instead.

Require one approach, not one shape

What operating model should we require of a firm - a central AI function, hub-and-spoke, or a capability in every unit?

Supervisors get asked to bless an operating model. I would resist. A shape that fits one firm fails the next, and you do not want to be in the business of approving ideals. The AIRG takes the same view and says so outright: consistency is required, the shape is not.

So the question I would put instead is whether one approach governs AI risk consistently across the whole firm. The spine. Not which box on the org chart it hangs from.

Read the inventory, then go to the low-rated

How do we know a firm has found all of its AI?

You cannot verify completeness from the outside, and chasing it will exhaust you. But you can read the inventory, and you can probe the ratings. The cheapest way a firm makes a lot of risk disappear is to rate it low, so a low rating is the one that quietly tells every other control to try less. When I inspected firms, I went to the systems rated low, not high.

So the question that tells you most is whether the firm knows where its AI is and has rated each use honestly, and whether the low ratings survive a second look.

Check that accountability is real, not a title

Should we require firms to appoint a Chief AI Officer?

A single accountable head is clean on paper. But hand one person accountability for all of AI and they may still not hold the levers, because the real controls sit in functions they do not run. Responsible for everything, able to move nothing. The financial sector solved this long ago, with the three lines of defence. AI slots into the structure you already supervise.

So the question I would put is whether someone is accountable for each AI outcome and actually holds the controls, across the three lines, not whether there is a Chief with AI in the title.

Ask for evidence, not a demo

Do our supervisors need to understand the models to supervise them?

A supervisor's job is to challenge, not to build. You already challenge credit and market-risk models without anyone in the room having built one, because the validation, the limits and the reporting are laid out in a form you can interrogate. AI is the same. The failure is rarely that a supervisor cannot understand. It is being handed a polished demo instead of an inventory, a materiality rating, and evidence that someone competent checked the work.

So the question that matters is whether the firm can show you evidence you can interrogate, including what "good enough" means for the task and the test that proves it, not whether your team can read the model's weights.

Set your scrutiny to the risk

How does supervision keep up with a technology that changes every few months?

The instinct is to chase the technology, a new expectation for every new model. But the task does not change. Fraud detection defines a good answer and a bad one whether it runs on a rule, a regression, or last week's model. Fix the standard to the task and it outlives the model. And you cannot supervise every AI use to the same depth, nor should you try. Match your scrutiny to materiality, the same way you expect firms to match their controls to it.

So the question that lasts is whether you are spending your supervisory attention where the risk actually is.

The rest of the book walks through these deeper.

Work with me

I train regulators, supervisors, and public authorities on AI governance and risk management. See the courses and workshops, read more on AI risk management, or get in touch.