Building AI supervisory capability

Building AI supervisory capability is the question that makes supervisors feel most behind, and it has two sides: whether the firm can keep running its controls as the technology moves, and whether you can keep supervising it. I supervised AI at the Monetary Authority of Singapore and wrote the AIRG, and the good news is that capability is a smaller problem than the panic suggests, once you stop framing it as a race against the technology.

How does anyone keep up with something that changes every few months? The models we supervised last year are obsolete. Our people will never move as fast as the labs.

The framing is the problem, because that race is unwinnable and you were never meant to run it. So the question turns: does the job the system does change every few months, or only the technology under it? Almost always, only the technology. Fraud detection is still fraud detection; customer advisory is still customer advisory. Fix your standard to the task, not the tool, and it outlives the model. That single move is what makes capability a manageable problem rather than a treadmill, for the firm and for you.

The firm's capability: enough to run its own controls

The AIRG expects a firm to have the competence and the capacity to run everything the lifecycle demands, and to resource it in proportion to the AI it has deployed. So you assess two things, and they are different. Competence is whether the people have the skills, across the three lines, not just in the build team. The second-line validators the whole framework leans on have to be able to actually challenge a model, which means real technical and domain skill, not a risk generalist signing forms. Staff using AI need enough literacy to use it within policy and to raise a concern when something looks wrong. Capacity is whether there are enough of them. A superb framework run by two people against three hundred systems is not capability; it is a bottleneck with good intentions, and the controls will quietly lapse under load. Count the people against the estate, check that the validators can really validate, and treat a competency or headcount gap as a finding in its own right, because it is the gap that will make every other control fail slowly.

Your capability: enough to challenge, not to build

Now the harder and more neglected half, your own team. The reflex is to think a supervisor must match the firm's technologists. Drop it. Your job is to challenge, not to build, and those need different things. You already challenge credit and market-risk models without anyone in the room having built one, because the validation, the limits, and the reporting are laid out in a form you can interrogate. AI is the same method under more pressure.

What your team needs is enough understanding of AI to know what good evidence looks like, what a thin answer sounds like, and where the risk hides: the properties that make it different, the materiality rating, the lifecycle controls, the specific failures to probe. That is learnable, and faster than people fear. What is scarce is the supervisory instinct: reading evidence, spotting the gap between the document and the practice, knowing when a confident demo is covering a hollow system. That you cannot buy off the shelf, and it is what makes a supervisor. So build the team around people who have it and teach them the AI, rather than hiring technologists and hoping they learn to supervise. A mixed team is the practical answer: a few people with real technical depth to go deep when a material system demands it, sitting with experienced supervisors who carry the instinct, so neither skill is a single point of failure.

Fix your standard to the task, not the tool, and it outlives the model.

Keeping up without chasing the technology

The way a supervisor stays current is not a new expectation for every new model. It is holding the standard to the task and letting the technology churn beneath it. Whether fraud detection runs on a rule, a regression, or last week's model, a good answer and a bad one are defined by the task, and the firm has to show the job still gets done to that standard with the AI in the mix. Anchor your expectations there and most of the churn stops mattering, because you are not supervising the model. You are supervising whether the firm has managed it, which is the same question it was three model generations ago. What does genuinely need refreshing is narrower than the panic suggests: the specific new failure modes a new class of system brings, the prompt-injection and jailbreak surface of generative systems, the action-space of agentic ones, so your probing stays sharp. That is a manageable, bounded kind of keeping up, not a race.

For the supervisor starting from scratch

A word for the emerging-market supervisor building a regime from a blank page, because the position is better than it feels. You are not behind; you are inheriting a settled set of questions with the labels still being argued over. The convergence between the main frameworks is your advantage: you can take one with standing, read it against the law you already enforce, and know the requirements barely move between them.

So start where the leverage is, not where the noise is. Do not begin by drafting a hundred pages of your own standards; begin by requiring firms to find their AI and rate it honestly, the inventory and the materiality rating, because that is the foundation everything else stands on, and the cheapest thing to get right early. Lean hard on proportionality, because it is what lets a small supervisory team cover a large estate: you were never going to look at everything, so build the regime around looking hard at what matters and auditing the firm's own sizing. Build the team around supervisory instinct and add the AI specifics. And borrow without embarrassment, because the frameworks, the guidance, and the comparisons are public, and a question that works for one supervisor works for most. A small, clear regime that asks the five questions well will out-supervise a large, elaborate one that asks the wrong ones.

Work with me

I train regulators, supervisors, and public authorities on AI governance and risk management. See the courses and workshops, read more on AI risk management, or get in touch.