Where AI risk hides from you

Risk does not hide where you are looking. It hides in three specific places, and your first line rarely puts it there on purpose. New second lines tend to spread their attention evenly, or point it at the biggest, newest models the board keeps asking about. That is understandable, and it is where the least risk is, because that is exactly where the business has already concentrated its own effort. I wrote the AIRG and led the thematic review of how banks actually manage AI model risk, so I have seen where AI risk hides from the second line. These are the three places.

We have limited people. Where do we spend them - the large models, the customer-facing ones, the generative stuff the board keeps asking about?

Those are where the business is already trying hardest, because they are where it knows you will look. So turn the question around:

Where would the risk sit if a busy first line wanted, without quite admitting it, to do less work?

Follow that and you find three places. None needs bad faith. A first line under commercial pressure drifts toward all three on its own, and the job of the second line is to know where they are and go there first, before the demo, before the showcase model. One of the three, you will notice, is your own programme.

The first hiding place: the low rating

Everything downstream hangs off one number - how material the use was rated. Rate it high and heavier controls follow: built more carefully, validated independently, watched more closely, reviewed more often. Rate it low and all of that is dialled down, legitimately. Which makes the rating the cheapest place in the whole system to make risk disappear. One quiet downgrade tells every control downstream to try less, and leaves no fingerprint.

So the materiality rating is where you spend scrutiny first, and you spend it the opposite way to instinct. Do not go to the systems rated high - those are already getting the heavy controls. Go to the ones rated low, and pull the thread on a sample. Why is this low? What would move it up a tier? Who in the business signed it, and did they benefit from it staying low? A first line whose low-rated pile is genuinely low will answer easily. A first line using the rating as a release valve will get vague, fast. There is a tell at the portfolio level too: if almost everything sits in the lower tiers, far more than a firm of your size and business should produce, that shape is itself the finding, and you are the one who has to raise it.

The second hiding place: the inventory gap

A control only reaches the AI that is written down. Everything off the inventory is ungoverned by definition - not badly governed, ungoverned - and invisible to you unless you go looking past the list you were handed.

You cannot prove completeness, and chasing it to the last system will exhaust you. But you do not need completeness. You need evidence the net is real. So do not audit the inventory line by line. Test the edges - the places AI enters the firm without passing through the front door: a feature switched on inside a tool the business already licenses, a vendor model embedded in a product nobody logged as AI, a spreadsheet that quietly became a model, a team using a public chatbot on personal accounts because the sanctioned route was slow. This is the shadow AI, and it is where the embarrassing failures come from, because no one chose to control what no one admitted was there. The net is yours to build, which is why the gap is yours to find.

A control only reaches the AI you wrote down.

The third hiding place: your own oversight on paper

The third place is the most uncomfortable, because it is you. A firm shows a governance framework, a committee with terms of reference, an approval workflow, a named owner. It looks like oversight. Whether it is oversight is a different question, and the documents will not answer it - and in the second line, that committee is often yours.

The gap is between the structure and whether the structure ever does anything. A committee that meets and approves everything put in front of it is not oversight; it is a stamp with a calendar. A validation function that has never sent a model back is a turnstile. An AI risk appetite written in words no one can breach is decoration. The quiet failure mode of a second line is to become exactly this: present at every gate, blocking at none, generating paper that lets everyone say the risk was managed. So look at your own function the way a supervisor would look at the firm. Can you point to a system your committee sent back, and what it demanded? A deployment your validation blocked or made conditional? A threshold that tripped and forced an action? Oversight leaves marks - rejections, conditions, escalations, things that did not ship. If your function has left none, it has not been tested, and that is on you, not the business.

Why these three, and why first

These are not the only places risk sits. They are where it hides, which is different. The visible, high-rated, well-documented AI is where the business's own effort already is. The low rating, the inventory gap, and the hollow committee are where effort can quietly be withdrawn while the paperwork stays intact. A second line that learns to go there first sees more of the real risk in an afternoon than a week of reviewing the showcase systems would reveal. And the third place is the one only you can fix, because no one outside will tell you your own oversight has gone hollow until something has already broken.

Work with me

I train risk and compliance teams on turning AI risk management into a working system. See the courses and workshops, read more on AI risk management, or get in touch.