Asking the database in plain language
Text-to-SQL, table question answering and semantic parsing appear on only 12 recent papers, all in AI journals, none in business ones. Firms are rolling out assistants that turn managers' questions into database queries. How often do the queries return the wrong numbers, and would a manager notice?
What prompted this#
Since October 2025, 12 papers carry a text-to-SQL, table-question-answering, semantic-parsing or natural-language-interface keyword, all in AI journals: "text-to-SQL" on 4, "table question answering" on 4, "semantic parsing" on 4. None appears in the management, accounting or operations research journals, where the users of business data publish.
Questions#
- On real company databases (many tables, ambiguous column names, business definitions such as "active customer"), how often do language models write queries that run but return the wrong answer?
- Which errors are most common: wrong joins, wrong filters, wrong definitions of a business term?
- Do managers who receive a number from an assistant check it, and does showing the query or a plain-language explanation change that?
Data that could help#
Public benchmarks (Spider, BIRD) and, more importantly, query logs and gold answers from a company data warehouse, plus user experiments.
Next step#
Build a test set of 200 business questions with gold answers on an anonymized company database and measure how often several models return a wrong but plausible number.