Foundation models for tables
Tabular data appears on 53 recent papers and tabular classification on 13, almost all in AI journals, and 16 tabular papers involve pretrained or language models. Most business data is tables: loans, transactions, patients, sales. Do pretrained tabular models beat gradient boosting on real business tables, and when?
What prompted this
Since October 2025, 77 papers carry a tabular keyword (73 in AI journals): "tabular data" on 53, "tabular data classification" on 13, "tabular data generation" on 4. Sixteen of them also carry a foundation-model, pretraining, transformer, language-model, in-context or zero-shot keyword. Thirteen were published in July–September 2026 against about 20 per quarter earlier.
Questions
- On credit, fraud, churn and insurance tables with their real sizes and missingness, do pretrained tabular models beat tuned gradient boosting, and by how much per unit of computing?
- Do the gains hold under the temporal distribution shift typical of business data, or only on random splits?
- Can a model pretrained on public tables transfer to a bank's private tables without leaking anything from the public ones?
Data that could help
Public business tables with time stamps (lending, fraud, insurance datasets), and private tables from a partner firm with temporal splits.
Next step
Benchmark two pretrained tabular models against tuned gradient boosting on five public business datasets using temporal rather than random splits.