iThink, therefore iCheck: Critical engagement with ChatGPT in linguistic analysis and learning

This study explores the integration of the free version of ChatGPT 4.0 into a graduate class, focusing on the tool’s ability to perform linguistic analysis and on students’ engagement with it through inductive learning. To address concerns about students’ uncritical use of ChatGPT, the research compares textual analyses of dialogs from two movies (drawn from the American Movie Corpus - (https://americanmoviecorpus.net) carried out by the instructors and by ChatGPT. Adopting a quasi-experimental design, it examines how two groups of graduate students of English – one trained in linguistic analysis and the multidimensional analysis (MDA) framework, the other untrained – interacted with ChatGPT’s analyses and evaluated both the tool and the learning experience. Both instructors and student groups used structured prompts to generate general textual and MDA-based analyses via ChatGPT. The instructors’ output (produced and collected at the same time as the students’ output) were analyzed to assess the tool’s performance, while the reflections on the experiment by the students served to evaluate the impact of prior training on their critical engagement. The findings reveal that ChatGPT’s ability to perform both general and MDA-based analyses was limited, often inconsistent and inaccurate. Students with prior MDA training showed stronger data literacy and more critical engagement with the tool, while untrained students exhibited overreliance and misconceptions regarding ChatGPT’s capabilities. These results highlight the need for targeted instruction to foster analytical skills and reduce uncritical AI use. This study contributes to ongoing debates on AI in education by underscoring the value of instructor guidance and structured training. It supports a pedagogical approach where AI is critically integrated into academic settings, encouraging informed and responsible student engagement.