Supervising third-party AI

Most of the AI in a firm, the firm did not build. It bought it, or it came switched on inside something else it bought. Supervising third-party AI is where the largest blind spot in AI supervision sits. I supervised AI at the Monetary Authority of Singapore and wrote the AIRG, and the instinct, firm and supervisor alike, is to treat bought AI as the vendor's problem. It is not, and the law you already enforce on outsourcing says so.

How can a firm be held responsible for a vendor's model it cannot see inside? The vendor will not hand over its weights or its training data.

Probably true, and beside the point. So the question turns: does buying the model instead of building it change who is accountable to you for the outcome, or only how hard the firm has to work to earn that accountability? Outsourcing the work does not outsource the responsibility. The AIRG is explicit: third-party AI sits inside the whole framework, identified, rated, inventoried, and controlled to the same standard as internal AI, with the firm doing compensatory testing to cover what the vendor will not show. A firm that can account fully for its own models and waves a hand at the bought ones has not reduced its risk. It has hidden most of it behind a contract.

The blind spot: bought AI off the map

Start where it goes wrong earliest, identification. Internal models get built by a team that knows they are building AI, so they tend to make the inventory. Bought AI arrives differently: embedded in a vendor product nobody logged as a model, switched on as a new feature in a tool the firm already licenses, delivered as an API that a business unit wired in without telling risk. So the same net test from the inventory applies with extra force here. Ask how procurement surfaces AI inside the things the firm buys, and how the firm keeps up when a vendor adds AI to a product that did not have it last year. The bought AI that is not on the inventory is not lightly governed; it is ungoverned, and it is usually the larger share.

The same standard, despite the information gap

Once a bought system is on the map, the principle is equivalence: it is held to the same governance, testing, fairness, and documentation standards as an internal model. The obvious objection is the information gap, the firm cannot inspect what it did not build. The AIRG's answer, and yours, is compensatory testing: the firm closes the gap from the outside.

This is the heart of supervising bought AI. The firm may not see the vendor's training data, but it can test the model on its own data, for its own population, against its own definition of good enough, the same task standard as everything else. It can monitor the vendor's model in production against a baseline and challenge it when it drifts. It can demand enough documentation to meet its own transparency obligations to customers and to you. So the test is not whether the firm trusts the vendor. It is whether the firm has done its own testing, on its own data, proportionate to how material the system is, or whether it has accepted the vendor's marketing as validation. A firm relying on a vendor's assurance that "the model is fair" or "the model is accurate," with no independent check on its own customers, has outsourced its validation along with the model, which it is not allowed to do.

A vendor's assurance is a claim to test, not a validation to accept.

The supply chain, and concentration

Bought AI is rarely a single vendor. Behind the product sits a model provider, behind that a foundation model, behind that a hosting platform and data sources and sub-processors. The firm should be able to map that chain for its material systems, because a failure or a change anywhere along it lands on the firm's customers.

And then the risk supervisors most often miss, because it is invisible from inside any single firm: concentration. When one vendor, or one underlying foundation model, sits behind critical functions across many firms, a single failure, a single bad update, or a single outage becomes a systemic event, not a firm-level one. At the firm level, ask whether one vendor underpins several of the firm's critical systems, and what the contingency is if that relationship fails. At the system level, yours, across the firms you supervise, keep a running sense of where the whole sector has quietly converged on the same provider. That is a financial-stability question, not a procurement one, and it is the FSB's 2026 concern in a sentence. No single firm will raise it for you.

Contracts and exit: the rights to supervise

The controls above only work if the firm has the contractual rights to exercise them. So check the contract, not for legal elegance but for the specific rights that make supervision and continuity possible. Audit rights, so the firm, and through it you, can actually examine the vendor rather than take its word. Clear allocation of liability when the model causes harm. Notice of material changes, so the vendor cannot silently swap the model under the firm's feet. And termination rights that can actually be used.

That last one is the exit question, and it is where "we are covered" most often turns out to be hollow. Ask whether the firm could actually leave this vendor, whether there is an alternative, whether the data and the function could transition without the business falling over, whether a continuity plan exists for the vendor going down or going away. A dependency you cannot exit is not a managed risk; it is a hostage situation with better paperwork. The firm that has never seriously asked how it would leave has not finished managing the relationship.

Work with me

I train regulators, supervisors, and public authorities on AI governance and risk management. See the courses and workshops, read more on AI risk management, or get in touch.