AI forecasting, with risk in mind
The risks of AI forecasting
A confident forecast is the dangerous one. Here is how to think about AI forecasting without being fooled by it.
AI has pushed forecasting forward, from classical statistical models through deep learning to the time-series foundation models arriving now. Used well, it can pick up structure a human would miss. Used badly, it produces a fluent, confident number that is wrong, and gives you no warning that it is wrong.
The protection is a way of thinking, not a particular model. What follows is the mental models I hold when forecasting with AI, not a how-to; for the methods and the agent that drives them, see the AI forecasting guide. This page is about not being fooled.
The mental models
- A forecast is a distribution, not a number. The uncertainty is the point. A single confident figure hides the range of what could happen, and the range is what you actually manage.
- The past is not the future. Models fail hardest at regime breaks, which is exactly when a forecast matters most. A method that fit forty years of history can still miss the turn.
- More parameters is not more accuracy. A bigger, more complex model overfits the past and mistakes noise for pattern. Complexity that improves the backtest often worsens the forecast.
- Watch for drift. The world moves, so a model that was right slowly becomes wrong. Monitor it against reality and re-validate on a schedule set by how much the decision depends on it; do not set and forget.
- A language model does not forecast on its own. It was trained on text, and a time series is a different kind of sequence. It can orchestrate the tools that forecast, and it can explain the result, but the computation belongs to the tools. Never mistake a fluent paragraph for a forecast.
- Calibration beats accuracy. A model that says "I am not sure" and is right about its uncertainty is worth more than one that is confident and wrong. Judge forecasts on whether their stated confidence holds up, not only on the headline hit rate.
- No method is reliably best. The right choice depends on the series in front of you. Match the method to the data, and prefer the simplest model that does the job.
Where this fits
- AI forecasting - how to build it: the ladder of methods, and the agent that drives the real forecasting tools.
- The complete guide to AI risk management in finance in Singapore - where this sits in the wider picture.
- The AIRG explained - a forecasting model is a model the guidelines cover.
- The research behind this, on Google Scholar.
I did a PhD building AI models that forecast financial markets from prices, news and networks, and I developed the MAS AIRG while leading AI risk supervision there. This is the risk-aware way to think about AI in forecasting. I am publishing more in the coming weeks; subscribe to get it.