AI infrastructure

AI infrastructure is the platforms, environments and tooling that AI runs on, and that AI risk management runs on. It is one of the seventeen areas in the MindForge AI Risk Management Toolkit, the Singapore industry's practices for the AIRG. MindForge is the practices; the AIRG is the expectations and standards. I wrote the AIRG, and this guide sets out the practice, the AIRG expectation it meets, and the evidence it produces.

What the AIRG expects

The AIRG expects a firm to have the capability and infrastructure to run its AI and its AI risk management, and to secure AI systems against technology and cyber risks. Infrastructure is where several of the lifecycle controls actually live: the environments AI deploys into, the access controls around it, and the tooling that produces the inventory, the monitoring and the guardrails. For how those controls fit together, see the AIRG and the AIRG in practice.

What MindForge says to do

The Toolkit is clear that most of this is existing technology and security good practice, applied to AI rather than reinvented for it. The AI-specific parts are what to pay extra attention to.

• Fit for purpose. Infrastructure sized for availability, scalability, data quality, security, privacy, and sustainability, because AI use cases carry high compute and data demands, on-premises or in the cloud.
• Access and segregation. Access management to AI components in line with existing security practice, including authentication and real separation of duties.
• Hardened deployment. Secure environments with identity and access management, input validation, encryption, and data-loss prevention, with particular care around third-party components and APIs.
• Measured differently. Generative AI needs usage and scaling metrics that traditional software does not, so capacity planning uses AI-appropriate measures.
• Capacity for how AI is built. Where advanced techniques are used (distributed, parallel, or federated approaches), the infrastructure needs the dependencies, networking and encryption to support them, and the ability to support governance features such as prompt gateways and caching.

The thread through all of it: the same infrastructure that runs AI is what lets you govern it consistently, rather than team by team.

How banks do it

The MindForge Implementation Examples show banks building an enterprise AI platform and a common data foundation, so AI can be deployed and governed to one standard across the firm rather than each team standing up its own environment.

My take

Infrastructure is what lets you govern a hundred systems the same way you govern one. Without it, you are governing by hand, which means you are not.

Most of this is your existing technology and security discipline pointed at AI. The mistake is to treat the platform as plumbing rather than as the thing that makes consistent governance possible at scale. (From my Governing AI at Scale primer.)

Work with me

I advise financial institutions on building the infrastructure and tooling to run and govern AI at scale under the AIRG. See the courses and workshops, read more on AI risk management, or get in touch.