Use-case design

The cheapest place to manage AI risk is at the start, when the use case is being designed. Getting the ownership, the go/no-go, and the human oversight right up front is one of the seventeen areas in the MindForge AI Risk Management Toolkit, the Singapore industry's practices for the AIRG. MindForge is the practices; the AIRG is the expectations and standards. This guide sets out the practice, the AIRG expectation it meets, and the evidence it produces. I wrote the AIRG, and oversight you bolt on after launch is almost never as good as oversight designed in.

What the AIRG expects

The AIRG expects human oversight that is proportionate to materiality and effective: the person can actually intervene, the system is built from the outset to enable it, and the arrangement accounts for automation bias and decision fatigue. A human who cannot stop the thing is not oversight. For how this fits the lifecycle, see the AIRG in practice and the AIRG explained.

What MindForge says to do

The Toolkit frames design as the first lifecycle stage, with four moves.

• Assign a use-case owner. One accountable owner from the start, including for use cases built on third-party AI, so governance tasks actually get done.
• Run a go/no-go. A preliminary check before resources are committed: who could be disadvantaged, whether it meets legal and regulatory requirements, and whether it aligns with the firm's values and AI principles. A use case clearly out of line may not be worth pursuing.
• Assess inherent risk early, and register it. A first materiality assessment to set the governance level, and an entry in the AI inventory as early as feasible, updated as the use case evolves.
• Design human oversight in. Decide at design time how much human oversight is desirable and practical, by materiality, modality and feasibility.

The three forms of human oversight

MindForge uses the forms from the national Model AI Governance Framework:

• Human in the loop - a person approves or executes at key points, such as reviewing an AI-drafted customer email before it is sent.
• Human over the loop - the AI acts, but a person can modify, overrule, or intervene, and errors are escalated for review.
• Human out of the loop - the AI acts without direct involvement, as with an autonomous chatbot. This does not remove oversight; it moves it to testing, monitoring and review after the fact.

Whichever form, the roles, responsibilities and escalations must be defined, and the people in them must have the competence and the authority to act. For high-throughput systems where real-time review is not feasible, oversight shifts to post-event monitoring and exception handling, chosen deliberately rather than by default.

How banks do it

In the MindForge Implementation Examples, Julius Baer uses a two-stage tollgate overseen by a cross-functional responsible-AI council. The first gate, at the end of ideation, screens out uses prohibited under the EU AI Act and flags high-risk ones; the second, by the end of development, is a holistic risk assessment that sets the stringency of the process, the controls, the oversight requirements, and the reassessment frequency before anything goes to production.

My take

A reviewer who cannot stop the thing, or who waves through a thousand a day, is not oversight. It is a scapegoat-in-the-loop.

Human oversight is the control most often claimed and least often real. Design it in, give the person the authority and the time to use it, or be honest that there is none and rely on monitoring instead. (From my book, AI Risk Management for Directors.)

Work with me

I train and advise financial institutions on designing AI use cases and human oversight that holds, and the rest of the AIRG programme. See the courses and workshops, read more on AI risk management, or get in touch.