Comparing the Efficacy of Artificial Intelligence Immersion and Human-Led Workshops for Enhancing Researchers' English Language Skills: A Randomized Control Trial

Abstract

Objective: Our study aimed to compare the efficacy of artificial intelligence (AI)-based immersive training with human-led workshops to improve the English language skills of non-English early career researchers (NEECRs) in a Nigerian public university. Methods: Our study employed a randomized pretest/posttest control group design. A total of 124 NEECRs in Federal University Oye-Ekiti, Nigeria, who met the eligibility criteria participated in the study. NEECR participants were randomly assigned to 8 weeks of self-directed AI immersion (<em>n </em>= 62) or human-led workshops (<em>n</em> = 62). A 56-item questionnaire collected data pretreatment, posttreatment, and at a 3-month follow-up. Repeated-measures ANOVA analyzed differences between and within the groups over time. Effect sizes were calculated using partial eta squared (ηp2). Prior to analysis, Mauchly’s test of sphericity was conducted to test the assumption of sphericity, yielding a nonsignificant result (<em>W</em> = 0.950, <em>p</em> = 0.247), indicating the assumption was not violated. Results: No initial group differences were found on pretest measures (all <em>p </em>> 0.05). At posttest, the AI group significantly outperformed the human-led group on all skills (<em>p </em>< 0.001, large effect sizes). For example, vocabulary scores were 22.1 for the AI group versus 20.4 for the human-led group. Similar significant results favoring the AI group were seen at the 3-month follow-up. Time-by-group interactions showed greater gains from AI workshops. Conclusions: AI-immersive instruction was found to be more effective in developing English proficiency in early career researchers compared with traditional human-led methods across all the domains measured. The findings suggested AI could help promote international scholars’ career advancement. Implications: Our study implicated the strategic use of AI to develop NEECRs’ English abilities. Institutions should consider incorporating AI-assisted language training to support internationalization goals and researcher career success in the English-dominated landscape of international scholarship. Policymakers could view the strategic incorporation of AI for language development favorably as a means to strengthen international competitiveness among researchers.