article · Qualitative Health Research
Large language models such as ChatGPT offer emerging possibilities for scientific inquiry, yet their utility in qualitative research remains underexplored. An evaluation comparing ChatGPT with an experienced researcher in analysing an interview transcript highlights notable parallels and distinctions. Themes generated across diverse prompts showed considerable overlap between the tool and the human analyst. Although the system favoured descriptive themes, it also detected nuanced interpersonal dynamics such as trust, responsibility, acceptance, and resistance. Furthermore, the model generated codebooks and extracted quotes displaying face validity, although these outputs still demand careful human review. The artificial intelligence also demonstrated an ability to connect identified themes to broader theoretical frameworks persuasively, even when using seemingly unfitting theoretical models. Overall, the tool exceeded performance expectations in theme identification and theoretical contextualisation, suggesting new ways to support qualitative workflows.
Qualitative research typically demands intensive manual analysis that is difficult to scale. Demonstrating that large language models can reliably extract themes, build preliminary codebooks, and articulate theoretical links offers researchers and educators an aid to speed up data processing, while also underlining the essential need for human review to evaluate validity and prevent the uncritical adoption of ill-fitting theoretical frameworks.
The abstract describes early-stage exploratory research testing the analytical capabilities of a general model on an interview transcript. The potential applications include assistive qualitative analysis and research-training tools for academic, educational, and social science researchers. Because outputs such as codebooks and quotes require thorough human review, this represents an early-stage methodology rather than a fully automated or near-market software product.
AI-generated from the published abstract. Always read the original work before citing.
The impact of ChatGPT and other large language model-based applications on scientific work is being debated across contexts and disciplines. However, despite ChatGPT's inherent focus on language generation and processing, insights regarding its potential for supporting qualitative research and analysis remain limited. In this article, we advocate for an open discourse on chances and pitfalls of AI-supported qualitative analysis by exploring ChatGPT's performance when analyzing an interview transcript based on various prompts and comparing results to those derived by an experienced human researcher. Themes identified by the human researcher and ChatGPT across analytic prompts overlapped to a considerable degree, with ChatGPT leaning toward descriptive themes but also identifying more nuanced dynamics (e.g., 'trust and responsibility' and 'acceptance and resistance'). ChatGPT was able to propose a codebook and key quotes from the transcript which had considerable face validity but would require careful review. When prompted to embed findings into broader theoretical discourses, ChatGPT could convincingly argue how identified themes linked to the provided theories, even in cases of (seemingly) unfitting models. In general, despite challenges, ChatGPT performed better than we had expected, especially on identifying themes which generally overlapped with those of an experienced researcher, and when embedding these themes into specific theoretical debates. Based on our results, we discuss several ideas on how ChatGPT could contribute to but also challenge established best-practice approaches for rigorous and nuanced qualitative research and teaching.
This page summarises published work. The authoritative version sits with the publisher.
DOI: 10.1177/10497323241244669
Is something wrong with this record? Report it or request removal.
Discussion
Have you built on this work, tried to replicate it, or seen it applied in practice? Share what you know. Verified researchers and MARATTO™ domain experts can open a discussion, and any member can reply. Contributions are reviewed before they appear.
No discussion yet. Open the first thread.
New to MARATTO™? Create a free account.
