AI security risks in finance
The deliberate-attack surface that AI adds, from injection and jailbreak to poisoning, extraction and evasion, and the controls that hold it. One risk from the wider taxonomy.
What it is
AI security risk is the deliberate-attack surface specific to AI. The family is worth knowing by name: injection, where hostile instructions arrive through the input, directly or planted in content the model retrieves; jailbreak, getting a model past its own guardrails; data poisoning, corrupting what the model learns from; model extraction, reconstructing a model through its outputs; and evasion, crafting inputs that get past it. Two properties make this a distinct discipline. Instructions and data share one channel, so a model cannot cleanly separate a command from content, and models are probabilistic, so the same defence does not hold every time. There is no clean fix; the aim is to reduce the risk, not remove it.
Why it matters in finance
An attacker does not need code. Hostile text in a document, a poisoned training source, or a crafted input can make a model misbehave, give away data, or, where it has tools, take an action. That last point is where security risk turns real: the moment text becomes an action, a manipulated model is no longer just wrong, it is doing something. In finance that something can move money, change a record or send a message.
What catches it
Security is the primary control, and it rests on least privilege: the model can reach only what the task needs, so even a successful attack cannot touch anything irreversible. Around that sit red-team testing that probes the real attack paths, monitoring for guardrail hits, vendor and supply-chain checks on the models and tools you did not build, data controls against poisoning, change management, and human oversight on high-impact actions. Good enough has a plain definition: the attack you missed cannot reach something you cannot undo.
What to ask
What is the worst thing this system could be made to do through its inputs? What are its permissions, and who set them? Has it been red-teamed against these attack types? What in the supply chain, models, data and tools, is trusted, and on what basis?
Related on this site: prompt injection risk, data leakage risk, governing agentic AI, governing third-party AI, and the guide for risk and control functions.
In an ebook: Agentic and Generative AI for Finance.
One risk from the wider set. See the full AI risks in finance taxonomy.