Do AI explanations agree with each other?
Explainable AI is one of the largest topics in the AI journals and is spreading into management, finance and economics. Explanation fidelity (whether an explanation reflects what the model really does) is a fast-rising but still small keyword. When SHAP, LIME and attention maps disagree about the same prediction, which should a loan officer or a doctor believe?
What prompted this
Since October 2025, "explainable AI" appears on 419 papers (341 in AI journals, 28 in management, 20 in operations research, 15 each in finance and economics). "Explanation fidelity" appears on 11 papers and is rising in the AI journals (6 papers in July–September 2026 against about 1 expected).
Questions
- How often do common explanation methods disagree about which features drive the same prediction?
- Which of them stay faithful when the model is changed in ways that should change the explanation?
- Do people decide better with explanations that are more faithful, or just with ones that look more confident?
Possible approach
Compare SHAP, LIME, integrated gradients and attention on standard tabular and text tasks, using tests that check fidelity by perturbing the model or its inputs.
Next step
Measure how often SHAP and LIME agree on the top features for one credit dataset and one medical dataset, across several model types.