AUTOMATED ESSAY SCORING AND FEEDBACK SYSTEMS FOR ESL LEARNERS: A META-REVIEW OF PEDAGOGICAL IMPACT

Abstract

This meta-review provides a comprehensive, quantitative synthesis of empirical research on the pedagogical impact of Automated Essay Scoring (AES) and Automated Writing Evaluation (AWE) systems for English as a Second Language (ESL) learners. Drawing from 54 primary studies published between 2000 and 2024, encompassing 7,832 participants across secondary, tertiary, and intensive English programs, the review investigates how automated scoring and feedback technologies influence writing performance, learner engagement, and assessment reliability. The study employed a systematic search across Scopus, Web of Science, ERIC, PsycINFO, ProQuest, and Google Scholar, guided by PRISMA and JARS-Quant frameworks to ensure methodological transparency and replicability. Quantitative data were analyzed using random-effects meta-analysis, robust variance estimation, and meta-regression to explore moderators such as learner proficiency, feedback frequency, delivery mode, and tool type (e.g., Criterion, Pigai, Grammarly, Write & Improve, and large language model–based systems). Results indicate that AES/AWE interventions produce significant improvements in writing quality, grammar accuracy, and lexical sophistication, with an average effect size of g ≈ 0.60, denoting a moderate pedagogical impact. Intermediate learners benefited most, while feedback frequency and immediacy emerged as strong predictors of performance gains. Systems demonstrating high alignment with human raters (ICC > .80) yielded the greatest learning improvements, highlighting AI precision as a crucial determinant of educational effectiveness. Engagement indicators—such as multiple draft cycles, higher feedback uptake, and reduced latency—further strengthened outcomes. However, fairness diagnostics and bias reporting were inconsistently addressed across studies, underscoring the need for more equitable validation frameworks in multilingual contexts. Overall, the findings affirm that when designed with psychometric rigor, timely feedback, and iterative revision opportunities, AES and AWE systems significantly enhance ESL writing development. This study contributes evidence-based insights for educators, developers, and policymakers, emphasizing that the pedagogical value of automated feedback lies not merely in automation itself but in its precision, transparency, and capacity to foster sustained learner engagement.