Language models as qualitative coders (research idea)

Language models as qualitative coders

Qualitative methods appear on 46 recent papers, mostly in management, and only 7 of them carry an AI or language-model keyword. Researchers are starting to let language models code interviews and open-ended answers. Do the codes agree with trained human coders, and what does the method section need to report?

What prompted this

Since October 2025, 46 papers carry a qualitative-research, qualitative-analysis, grounded-theory, interview, focus-group or qualitative-comparative-analysis keyword (26 in management journals): "fuzzy-set qualitative comparative analysis" on 25, "qualitative research" on 4, "expert interviews" on 4, "grounded theory" on 2. Only 7 also carry an AI or language-model keyword. A related note asks whether models can stand in for survey respondents; this one is about models reading what real respondents said.

Questions

On interview transcripts already coded by trained researchers, how closely do language-model codes agree, by code and by how abstract the code is?
Do models miss what is unusual (the rare theme that changes the story) more than what is common?
What should authors report so readers can judge model-assisted coding: prompts, model versions, agreement statistics, human checks?

Data that could help

Published qualitative datasets with codebooks and human coding (qualitative data archives such as the UK Data Service), and journal guidelines on AI use.

Next step

Re-code a publicly archived interview study with two language models using the original codebook and compare agreement with the original coders code by code.

Related research activity

The keywords this note follows, in the tracked finance, accounting, economics, management, operations research and AI journals as of 2026-10-26
Keyword	Papers, last 30 days	Rank	Papers, last 36 months
qualitative research	0	–	11
fuzzy-set qualitative comparative analysis	2	#1303	55
grounded theory	0	–	4

@misc{scholar4ai-ideas-llms-as-qualitative-coders, author = {{Scholar4AI}}, title = {Language models as qualitative coders}, year = {2026}, howpublished = {\url{https://scholar4ai.com/ideas/llms-as-qualitative-coders}}, urldate = {2026-10-26}, note = {Journal Research Rankings; data from OpenAlex (CC0); licensed CC BY-NC 4.0} }