Quaintitative

AI Risk Management

AI risk management, from first principles

AI risk management is the practice of finding where AI is used, judging how much risk each use carries, and putting controls in place across the AI lifecycle so the risk stays within what an organisation is willing to accept. It is not the same as using AI to manage other risks. It is the discipline of governing the risks that AI itself creates. I developed Singapore's AI risk management guidelines for the financial sector while leading AI risk supervision at the Monetary Authority of Singapore, and this page sets out how I think about it.

What most AI risk management gets wrong

Someone will try to sell you AI governance that looks impressive: a lofty set of principles, a thousand-question control checklist, or a certification with a handsome logo. Most of it will not actually manage anything. A principle tells you what an organisation hopes for. It does not tell anyone what to build or what to check. The test for whether a control is real is simple: is it clear, and does it have an owner, a threshold that triggers action, and a test that runs? If those are missing, nothing is happening, however long the document. A brief useless checklist and a long useless one are both still useless.

The three questions at its core

When you strip AI risk management down to what matters, across every framework and sector, it is three questions:

  • What's at risk? Find where AI is used, and judge how much each use could hurt.
  • How do we manage it? Put controls in the areas that matter, and check that they work.
  • Who's accountable? Name who answers for the outcome when the AI is wrong.

Find what's risky, manage it, own it. Every AI governance or risk framework I have read asks these three under different names.

Why AI needs its own risk management

AI risk management extends decades of model risk and technology risk management, but one thing makes it different. Traditional software is a list of rules someone wrote down. With AI, nobody writes the rule: you hand a model past data and it works out its own function, with its behaviour spread across weights nobody chose by hand. There is no rule anyone can read, so validation shifts from checking the logic to probing the behaviour.

It is worth being plain about what these systems are. A model is a mathematical mapping from inputs to outputs. It does not understand, intend, scheme, or think like a human. Treating a fitted function like a person, in either direction, is the fastest way to misread its risks: trust it like a colleague and you over-trust it; fear it like a villain and you hunt for motives when the cause is a skewed training set.

What an AI risk management programme covers

Underneath the three questions, every framework has the same four parts, working as a loop rather than a checklist:

  • Oversight and accountability. Board and senior management answer for how AI risk is managed across the organisation.
  • Risk management systems. Identifying where AI is used, keeping a live inventory, and rating each use's materiality on impact, complexity and reliance, so effort follows risk.
  • Lifecycle controls. Data management, fairness, transparency and explainability, human oversight, third-party AI, evaluation and testing, security, monitoring, and change management.
  • Capability and capacity. The skills and infrastructure to run all of the above.

These interlock. Pull a control out and the ones that leaned on it should visibly fail. For how the parts work as one system, see Governing AI at Scale. For putting the inventory, materiality and controls to work, see the AIRG in practice; for AI that acts on its own, see governing agentic AI.

The main frameworks

Several frameworks describe AI risk management, in different forms and with different force: the EU AI Act (binding law), the US NIST AI Risk Management Framework (voluntary), ISO/IEC 42001 (a certifiable management-system standard), and, for finance, the MAS AI Risk Management Guidelines. They diverge in packaging but converge on the same underlying areas, so work done for one maps substantially onto the others. For a side-by-side, see how the AIRG compares to the EU AI Act, NIST and ISO 42001.

AI risk management for financial institutions

Finance came to this early, because it already treated models as a source of risk. Singapore's AI Risk Management Guidelines (AIRG) set supervisory expectations for financial institutions across the same areas above, applied in proportion to risk. If you are a bank, insurer, or asset manager, that is the version that applies to you.

Where to start

You do not start with a hundred-page policy. You start by finding your AI and rating which uses are material, then applying heavier controls where the risk is highest. A short, honest inventory is worth more than a long, aspirational framework.

Work with me

I train and advise financial institutions and their boards on AI risk management, from first principles to implementation. See the courses and workshops, or get in touch.