Does the model know when it is guessing?
Confidence calibration and uncertainty calibration appear on 54 recent papers, mostly in AI journals, and 35 calibration papers involve language or vision-language models, with 94 calibration-related papers in the last quarter against about 57 per quarter before. When a model answers a financial or legal question, its stated confidence rarely matches its accuracy. Can calibration be fixed, and do users read it?
What prompted this
Since October 2025, 280 papers carry a calibration or overconfidence keyword (176 in AI journals, 53 in finance, 28 in economics): "confidence calibration" on 28, "uncertainty calibration" on 26, "model calibration" on 19, besides the behavioural overconfidence of executives and investors studied in finance. Thirty-five of the 280 also carry a language-model or vision-language keyword. The quarter to September 2026 had 94 such papers against about 57 per quarter earlier.
Questions
- On financial, accounting and legal questions, how far are models' stated confidences from their accuracy, and does the gap grow for harder questions?
- Which methods (verbalized confidence, sampling consistency, trained calibrators) give usable confidence for decisions, and do they survive fine-tuning?
- When users are shown calibrated confidence, do they rely on the model more where it is right and less where it is wrong?
Data that could help
Question sets with gold answers in finance and law, model outputs with confidence under several methods, and user experiments with confidence displays.
Next step
Measure the calibration of three models on a financial analysis question set under verbalized and consistency-based confidence, and test whether showing it changes user reliance.