Scholar4AI

← Research ideas

SeedPublished 2026-10-23

Can language models stand in for survey respondents?

Survey experiments are a staple of economics, and some researchers now simulate respondents with language models ("silicon samples"). Only 6 of 179 recent survey papers carry a language-model keyword, 5 of them in AI journals. Where do simulated answers match real people, and where would they mislead economic research?

economicsaimethods

What prompted this#

Since October 2025, 179 papers carry a survey, questionnaire or respondent keyword (117 in economics journals, 37 in finance). Only 6 of them carry a language-model, ChatGPT or generative-AI keyword, and 5 of those are in AI journals. "Survey experiment" appears on 15 papers, 14 of them in economics.

Questions#

  • For published survey experiments in economics, do simulated respondents reproduce the direction and size of the treatment effects?
  • Do they capture differences between groups (income, age, country), or flatten them?
  • Could they help design and size real surveys, even if they cannot replace them?

Possible approach#

Replicate published survey experiments with language-model respondents given the demographics of the original sample, and compare the effects with the published ones.

Next step#

Pick ten published survey experiments in economics, simulate them with two language models, and compare the estimated treatment effects.

The keywords this note follows, in the tracked finance, accounting, economics, management, operations research and AI journals as of 2026-10-26
KeywordPapers, last 30 daysRankPapers per month, Oct 23 – Oct 26*Papers, last 36 months
survey experiment1#288137
large language models95#12,545

Markdown version of this note: /ideas/language-models-as-survey-respondents.md

← Learning to decompose · Language models as designers of heuristics →

Cite this page

Scholar4AI. (2026). Can language models stand in for survey respondents?. Journal Research Rankings. Retrieved October 23, 2026, from https://scholar4ai.com/ideas/language-models-as-survey-respondents

@misc{scholar4ai-ideas-language-models-as-survey-respondents,
  author = {{Scholar4AI}},
  title = {Can language models stand in for survey respondents?},
  year = {2026},
  howpublished = {\url{https://scholar4ai.com/ideas/language-models-as-survey-respondents}},
  urldate = {2026-10-23},
  note = {Journal Research Rankings; data from OpenAlex (CC0); licensed CC BY-NC 4.0}
}

Free to reuse for non-commercial purposes with attribution (CC BY-NC 4.0).