Governing agentic AI

Agentic AI governance is about controlling AI that takes actions on its own, not just AI that produces an output. You cannot govern an agent by praying it will behave. You govern it by shaping where it can go, cutting some paths and constraining others, and by checking that the shape is holding. I wrote Singapore's AI risk guidelines and contributed to MAS's work on safeguards for agentic finance, and this is how I think about governing agents.

Why agents are different from models

A normal model gives you one output for one input. You can feed it a thousand test cases, score the answers, and the thing you tested is the thing you deployed. An agent chooses an action, then another, and each choice opens more. The output you might eventually check sits at the end of a path, but the path itself matters. A refund of the right amount, issued to the wrong account, after skipping the verification step, is a fine-looking output on a terrible route. And when a normal model fails, it usually fails where you are looking. When an agent fails mid-path, the later steps can paper over it: the agent recovers, improvises, writes a fluent summary, and the final answer looks fine.

You cannot check every path

Picture an agent as a tree of paths, each branch a route it might take. An agent that resolves a billing dispute in six steps, with a handful of choices at each step, is already into the thousands of distinct paths. Nobody reviews thousands of paths; nobody even lists them. So anyone selling "fully autonomous, fully safe" is overselling it. But you do not govern an agent by inspecting every branch, any more than a gardener inspects every branch of a bonsai. You shape it.

Sort the branches

Not all branches are equal, and the structure is where governance starts. Sort them two ways.

• By impact. Some paths are truly bad: real, irreversible harm, such as refunding to the wrong account or switching off the audit log. Some are potentially bad but fine if controlled, such as a large refund with sign-off. Many are low stakes and reversible, such as the wording of a reply.
• By controllability. Some actions can genuinely be constrained with a rule: a recency check, a whitelist of sources, a hard ceiling on an amount. Others need judgment: filtering open-ended advice, judging tone, deciding whether a complaint is genuine.

Plot impact against controllability, and the treatment for each branch follows.

Four ways to govern an agent

• Cut. Remove the branch entirely. The high-impact paths that should never be reachable, such as exporting the full customer list or disabling the log, are not constrained; they are removed. A branch that does not exist cannot go wrong, which is why cutting beats everything else.
• Rigid wire. Where a rule can genuinely hold it, wire the action: a hard ceiling on an amount, a whitelist, a required verification step. A rule belongs on the action, not on the judgment, and it adds no new paths.
• Supple wire. Where judgment is needed, you are growing a second tree: a judge, often another model, that checks the first. That judge is itself an AI system, so it has to be governed too, not trusted just because it hands you a verdict.
• Your own hands. Evaluation and testing, aimed at the path, not just the answer. Repeat everything, because agents are non-deterministic. Probe through everything the agent reads, because its inputs are an attack surface. And treat every surprise as a wire waiting to be added.

Not every agent needs the full treatment

Governing agents is still risk-proportionate. A low-stakes internal summariser is not a payments agent, and treating them the same is the checklist reflex again. Match the depth of the controls to the agent's impact and how much you rely on it, and revisit as the agent grows, because it will.

How this fits the AIRG

The AIRG covers agents within its lifecycle controls, applied in proportion to risk. MAS's Safeguards for Agentic Finance at Runtime (SAFR), which I contributed to, focuses on runtime controls for agentic systems in finance. Both sit on the same foundation as AI risk management generally and Governing AI at Scale: know what is risky, control it in proportion, and keep someone accountable.

Work with me

I train and advise financial institutions and their boards on governing agentic AI, from the first pilot to production. See the courses and workshops, or get in touch.