---
title: "Small models for finance (research idea)"
description: "Knowledge distillation, model compression and small language models appear on more than 700 recent papers in AI journals, and only 12 of them concern finance, banking, legal or business tasks. Banks cannot send customer data to the largest models, and regulators want models they can inspect. How small can a model be and still do financial work, and what is lost in the shrinking?"
url: https://scholar4ai.com/ideas/small-models-for-finance
markdown_url: https://scholar4ai.com/ideas/small-models-for-finance.md
license: CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)
date: "2026-10-23"
updated: "2026-10-23"
status: "seed"
tags: ["ai", "finance", "methods"]
keywords: ["knowledge distillation", "small language models", "model compression"]
---

# Small models for finance

Knowledge distillation, model compression and small language models appear on more than 700 recent papers in AI journals, and only 12 of them concern finance, banking, legal or business tasks. Banks cannot send customer data to the largest models, and regulators want models they can inspect. How small can a model be and still do financial work, and what is lost in the shrinking?

## What prompted this

Since October 2025, 763 papers carry a knowledge-distillation, model-compression, small-language-model, quantization, pruning or parameter-efficient keyword (748 in AI journals): "knowledge distillation" on 345, "model compression" on 114, "parameter-efficient fine-tuning" on 111, "small language models" on 17. Only 12 of the 763 also carry a finance, bank, stock, fraud, credit, economics, legal, accounting, business or customer keyword.

## Questions

- On tasks such as sentiment, entity extraction from filings and customer intent classification, how much accuracy is lost when a large model is distilled to one that runs on a bank's own servers?
- Do small models fine-tuned on a firm's data beat large general models on that firm's tasks, and for how long before the data drifts?
- Do compressed models fail in different places than their teachers, in ways that matter for compliance?

## Data that could help

Public financial NLP benchmarks (Financial PhraseBank, FiQA, FinQA), open models at several sizes, and a partner firm's task data.

## Next step

Distil a large model into three smaller ones on financial sentiment and filing extraction tasks, and map where the small models' errors differ from the teacher's.

## Related research activity

*The keywords this note follows, in the tracked finance, accounting, economics, management, operations research and AI journals as of 2026-10-26*

| Keyword | Papers, last 30 days | Rank | Papers per month, Oct 23 – Oct 26* | Papers, last 36 months |
|---|---:|---:|---|---:|
| [knowledge distillation](https://scholar4ai.com/keywords/knowledge-distillation.md) | 23 | #18 | 0, 0, 0, 0, 0, 0, 18, 15, 12, 21, 26, 16, 17, 17, 21, 31, 22, 32, 25, 27, 23, 26, 24, 21, 28, 33, 37, 51, 31, 33, 34, 52, 32, 29, 25, 32, 17 | 811 |
| small language models | 1 | #2881 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 1, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 3, 1, 1, 2, 3, 0, 1, 1, 2, 2, 1, 3, 1, 2, 0 | 36 |
| model compression | 5 | #393 | 0, 0, 0, 0, 0, 0, 9, 6, 8, 9, 12, 6, 5, 6, 8, 9, 4, 9, 17, 11, 10, 10, 7, 7, 6, 14, 10, 13, 10, 10, 11, 16, 9, 15, 11, 8, 3 | 286 |

## Cite this page

Scholar4AI. (2026). Small models for finance. Journal Research Rankings. Retrieved October 23, 2026, from https://scholar4ai.com/ideas/small-models-for-finance

```bibtex
@misc{scholar4ai-ideas-small-models-for-finance,
  author = {{Scholar4AI}},
  title = {Small models for finance},
  year = {2026},
  howpublished = {\url{https://scholar4ai.com/ideas/small-models-for-finance}},
  urldate = {2026-10-23},
  note = {Journal Research Rankings; data from OpenAlex (CC0); licensed CC BY-NC 4.0}
}
```

License: CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/).