Third-party AI

Most of the AI in your firm, your firm did not build. It bought it, or it came switched on inside something else it bought. Third-party AI risk is the largest blind spot in the second line's view, and it is also the one that extends most directly from work you already do. I wrote the AIRG and led the thematic review of how banks actually manage AI model risk, and the vendor AI is where I most often found the gap. The instinct across the firm is to treat bought AI as the vendor's problem. It is not, and the outsourcing rules your function already runs say so.

How can we be responsible for a vendor's model we cannot see inside? They will not hand over the weights or the training data.

Probably true, and beside the point. So the question turns: does buying the model instead of building it change who answers for the outcome, or only how hard you have to work to be able to answer? Outsourcing the work does not outsource the responsibility. Third-party AI sits inside your whole framework, identified, rated, inventoried, and controlled to the same standard as internal AI, with compensatory testing to cover what the vendor will not show. This is not new ground for you. It is your existing third-party and vendor-risk discipline, stretched to cover a product whose behaviour lives in data you cannot see. A firm that accounts fully for its own models and waves a hand at the bought ones has not reduced its risk. It has hidden most of it behind a contract, and the contract is yours to read.

The blind spot: bought AI off the map

Start where it goes wrong earliest - identification. Internal models get built by a team that knows it is building AI, so they tend to reach your inventory. Bought AI arrives differently: embedded in a vendor product nobody logged as a model, switched on as a new feature in a tool the firm already licenses, delivered as an API a business unit wired in without telling you. So the net you built for the inventory has to stretch here with extra force. Wire procurement and vendor onboarding so AI inside the things the firm buys surfaces to you, and keep a way to catch it when a vendor adds AI to a product that did not have it last year. The bought AI that is not on your inventory is not lightly governed. It is ungoverned, and it is usually the larger share.

The same standard, despite the information gap

Once a bought system is on the map, the principle is equivalence: you hold it to the same governance, testing, fairness, and documentation standards as an internal model. The obvious objection is the information gap - you cannot inspect what you did not build. The answer is compensatory testing: you close the gap from the outside, and this is your job, not the vendor's.

This is the heart of governing bought AI. You may not see the vendor's training data, but you can test the model on your own data, for your own population, against your own definition of good enough - the same task standard as everywhere else. You can monitor the vendor's model in production against a baseline and challenge it when it drifts. You can demand enough documentation to meet your own obligations to customers and to the supervisor. So the test is not whether you trust the vendor. It is whether you have done your own testing, on your own data, proportionate to how material the system is - or whether you have accepted the vendor's marketing as validation. Leaning on a vendor's assurance that "the model is fair" or "the model is accurate," with no independent check on your own customers, outsources your validation along with the model, and that is the one thing you cannot outsource.

A vendor's assurance is a claim to test, not a validation to accept.

The supply chain, and concentration

Bought AI is rarely a single vendor. Behind the product sits a model provider, behind that a foundation model, behind that a hosting platform and data sources and sub-processors. For your material systems you should be able to map that chain, because a failure or a change anywhere along it lands on your customers, and on you.

Then the risk that is easy to miss from inside a single desk: concentration. When one vendor, or one underlying foundation model, sits behind several of your critical functions, a single failure, a bad update, or an outage stops being one incident and becomes many at once. Ask which of your critical systems lean on a single vendor or a single foundation model, and what the contingency is if that relationship fails. This is also the thing the supervisor and the board will ask you about, because it is where the whole sector has quietly converged on the same few providers, so get ahead of it. It is your existing concentration-risk discipline pointed at a new kind of supplier.

Contracts and exit: the rights you need

The controls above only work if you have the contractual rights to exercise them, so read the contract for the specific rights, not the legal elegance. Audit rights, so you can actually examine the vendor rather than take its word. Clear allocation of liability when the model causes harm. Notice of material changes, so the vendor cannot silently swap the model under your feet. And termination rights you could actually use.

That last one is the exit question, and it is where "we are covered" most often turns out to be hollow. Ask whether the firm could actually leave this vendor - whether there is an alternative, whether the data and the function could transition without the business falling over, whether a continuity plan exists for the vendor going down or going away. A dependency you cannot exit is not a managed risk. It is a hostage situation with better paperwork. If no one has seriously asked how the firm would leave, the relationship is not finished being managed, and getting those rights into the contract is cheapest before you sign, which is why procurement is a second-line checkpoint, not a formality.

Work with me

I train risk and compliance teams on turning AI risk management into a working system. See the courses and workshops, read more on AI risk management, or get in touch.