Small models for finance
Knowledge distillation, model compression and small language models appear on more than 700 recent papers in AI journals, and only 12 of them concern finance, banking, legal or business tasks. Banks cannot send customer data to the largest models, and regulators want models they can inspect. How small can a model be and still do financial work, and what is lost in the shrinking?
What prompted this
Since October 2025, 763 papers carry a knowledge-distillation, model-compression, small-language-model, quantization, pruning or parameter-efficient keyword (748 in AI journals): "knowledge distillation" on 345, "model compression" on 114, "parameter-efficient fine-tuning" on 111, "small language models" on 17. Only 12 of the 763 also carry a finance, bank, stock, fraud, credit, economics, legal, accounting, business or customer keyword.
Questions
- On tasks such as sentiment, entity extraction from filings and customer intent classification, how much accuracy is lost when a large model is distilled to one that runs on a bank's own servers?
- Do small models fine-tuned on a firm's data beat large general models on that firm's tasks, and for how long before the data drifts?
- Do compressed models fail in different places than their teachers, in ways that matter for compliance?
Data that could help
Public financial NLP benchmarks (Financial PhraseBank, FiQA, FinQA), open models at several sizes, and a partner firm's task data.
Next step
Distil a large model into three smaller ones on financial sentiment and filing extraction tasks, and map where the small models' errors differ from the teacher's.