Where firms hide AI risk

Risk does not usually hide where a supervisor is looking. It hides in three specific places, and a firm rarely puts it there on purpose. New supervisors tend to spread their attention evenly, or point it at the biggest, most visible systems. That is understandable, and it is also where the least risk is, because that is exactly where the firm has concentrated its own effort. I supervised AI at the Monetary Authority of Singapore and wrote the AIRG; these are the three hiding places I learned to go to first.

If we only have so many hours, where do we spend them? The large models? The customer-facing ones? The newest ones everyone is talking about?

Those are where the firm is already trying hardest, because they are where it fears you will look. So turn the question around: where would the risk sit if a firm wanted, without quite admitting it, to do less work? Follow that and you find three places. None requires bad faith. A busy firm under commercial pressure drifts toward all three on its own.

The first hiding place: the low materiality rating

Everything a firm does downstream hangs off one number: how material it rated the system. Rate a use high and heavier controls follow, built more carefully, validated independently, watched more closely, reviewed more often. Rate it low and all of that is dialled down, legitimately. Which makes the rating the cheapest place in the whole system to make risk disappear. One quiet downgrade tells every control downstream to try less, and leaves no fingerprint.

So spend your scrutiny here first, and spend it the opposite way to instinct. Do not go to the systems rated high; those are being looked after. Go to the ones rated low and pull the thread on a sample. Why is this low? What would it take to move it up a tier? Who signed the rating, and were they independent of the people who benefit from it staying low? A firm whose low-rated pile is genuinely low will answer easily. A firm using the rating as a release valve will get vague, fast. There is a tell at the portfolio level too: if almost everything sits in the lower tiers, far less high than you would expect for a firm of that size and business, that shape is itself a finding. Make them show you which.

The second hiding place: the inventory gap

A control only reaches the AI the firm has written down. Everything off the inventory is ungoverned by definition, not badly governed, ungoverned, and invisible to you unless you go looking past the list you were handed.

You cannot verify completeness from the outside, and chasing it to the last system will exhaust you. But you do not need completeness. You need evidence the net is real. So do not audit the inventory line by line. Test the edges, the places AI enters a firm without passing through the front door: a feature switched on inside a tool the business already licenses, a vendor model embedded in a product nobody logged as AI, a spreadsheet macro that quietly became a model, a team using a public chatbot on a personal account because the official process was slow. This is the shadow AI, and it is where the embarrassing failures come from, because no one chose to control what no one admitted was there. Ask the firm not what is on the inventory, but how it finds what is not.

A control only reaches the AI the firm wrote down.

The third hiding place: oversight on paper

The third place is the most polished, and the easiest to accept if you are tired. A firm shows you a governance framework, a committee with terms of reference, an approval workflow, a named owner. It looks like oversight. Whether it is oversight is a different question, and the documents will not answer it.

The gap is between the structure and whether the structure ever does anything. A committee that meets and approves everything put in front of it is not oversight; it is a stamp with a calendar. An accountable owner who does not hold the levers is a name on a chart. A risk appetite for AI written in words no one can breach is decoration. So look past the existence of the structure to its friction. Show me a system this committee sent back, and what it demanded. Show me a deployment the independent function blocked or made conditional. Show me an override that changed a model, a breach that tripped a threshold and forced an action. Oversight leaves marks. Where there are no marks, there is probably no oversight, however good the paperwork. A structure that has never said no has never been tested.

Why these three, and why first

These are not the only places risk sits. They are where it hides, which is different. A firm's visible, high-rated, well-documented AI is where its own effort already is. The low rating, the inventory gap, and the hollow committee are where effort can quietly be withdrawn while the paperwork stays intact. A supervisor who learns to go there first, before the demo, before the big model everyone wants to show off, sees more of the real risk in an afternoon than a week of touring the showcase systems would reveal.

Work with me

I train regulators, supervisors, and public authorities on AI governance and risk management. See the courses and workshops, read more on AI risk management, or get in touch.