AI‐Supported Socratic Dialogue Agents vs. Generic AI Chatbots for Intermediate EFL Learners’ Critical Thinking, Epistemic Beliefs, and Argumentation: Mixed‐Methods Evidence

ABSTRACT

This mixed‐methods, quasi‐experimental study investigated the impact of artificial intelligence (AI)‐supported Socratic Dialogue Agents (SDAs) on critical thinking, epistemic beliefs, and argumentation skills among intermediate English as a Foreign Language (EFL) learners. Ninety male intermediate EFL learners from Iranian language institutes were randomly assigned to one of three conditions: (a) an AI‐Socratic dialogue agent group, (b) a generic AI chatbot group, or (c) a no‐AI control group. Quantitative data were collected through validated instruments before and after the intervention and then analyzed using mixed‐design analyses of variance (ANOVAs). To complement these results, qualitative data from post‐intervention focus groups (five participants per group) were examined through thematic analysis to elicit perceptions and through content‐structural analysis to examine real‐time English interactions. The quantitative findings indicated that the AI‐Socratic group achieved notably greater gains in critical thinking, epistemic beliefs, and argumentation skills than the generic AI and control groups, while the qualitative findings helped to account for these outcomes by foregrounding greater cognitive challenge, an enhanced understanding of knowledge, the cultivation of advanced argumentation strategies, and sustained motivational engagement within the AI‐Socratic condition; taken together, the findings carry theoretical implications in suggesting that AI‐supported Socratic dialogue can embody principles of constructivist and epistemic scaffolding.