Model drift in AI
A validation is a moment; production keeps moving. Why a model decays with no signal after it ships, and how you see it early. One risk from the wider taxonomy.
What it is
A validation is a moment; production keeps moving. Model drift is when the world moves away from the conditions the model was built for, and performance decays. It comes in three kinds: data drift, where the inputs change; concept drift, where the relationship between the inputs and the right answer changes; and model drift as the combined effect on what the system actually does. The hard part is that it fails without any signal. There is nothing to say an answer is now wrong, and for many models the true outcome arrives late, so the earliest sign is in the inputs, not the results. Input drift is a leading indicator: it tends to move before accuracy does.
Why it matters in finance
A credit, fraud or pricing model that is reviewed once a year but used every month keeps making decisions in between, and a decay no one can see keeps being acted on until someone notices. Post-deployment is where the risk actually shows up, and monitoring is the control with no natural owner, so it is the one most often left thin.
What catches it
Monitoring against a baseline, with statistical measures for how far the live data has moved from it. Two thresholds, like circuit breakers: a warning and a breach, each wired to a defined response, because an indicator no one acts on is just a statistic, and the real weakness is usually the response gap, not the detection. A kill switch that someone owns and has tested. Cadence set by the risk rating, clear revalidation triggers, champion-challenger comparison, and a golden set to catch a vendor model that changes without notice.
What to ask
What is monitored, against what baseline, and how often? Is input drift watched, or only outcomes? What are the thresholds, and what actually happens when one is breached? Who owns the monitoring, and how would the firm know if a vendor updated the model without notice?
Related on this site: monitoring, in full, testing, evaluation and assurance, reproducibility and traceability, third-party model concentration, and the guide for risk and control functions.
In an ebook: AI Risk Management for Risk and Compliance.
One risk from the wider set. See the full AI risks in finance taxonomy.