Cynthia Kay, MD, MS; Sarah Vepraskas, MD; Hari Paudel, MD; Marika I. Wrzosek, MD, JD; Leslie Ruffalo, PhD, MS; Kathleen Beckmann, MD, MPH; Carley Sauter, MD; Mary Ann Gilligan, MD; Heather Aschoff, BA
WMJ. 2026;125(4):492-497. Published November 1, 2026.
ABSTRACT
Introduction: The use of generative artificial intelligence (AI) for medical education research remains controversial. Although AI has the potential to revolutionize qualitative research analysis, concerns regarding its validity and accuracy remain barriers to its use in research. We sought to compare AI-generated qualitative coding with human coding.
Methods: As part of a mandatory course, third-year medical students watched 2 health care-related videos and responded to prompts on a virtual discussion board. Students’ posts were anonymized. Human coders met multiple times over a 4-month period to create a code list, generate themes, and produce an explanatory model. Generative AI also was asked to identify the main themes. Three external reviewers with expertise in qualitative research compared each pair of human-coded and AI-generated themes.
Results: Human coders required nine 60-minute meetings to code students’ posts, whereas AI completed the analysis in approximately 30 seconds. Consensus among the external reviewers was that the pairs of themes were not the same or similar.
Conclusions: AI required substantially less time and effort than human coders. Although most reviewers concluded that AI-generated themes were not comparable to human-generated themes, this finding was not unanimous. Generative AI may facilitate and accelerate qualitative research in medical education; however, concerns regarding the validity and contextual interpretation of AI-generated analyses must be addressed before its widespread use in qualitative medical education research.