Computer Assisted Language Learning

Beyond Automation: AI-Driven Language Assessment in CALL through Pedagogical, Ethical and AI Literacy Lenses

Language assessment plays a central role in measuring learners’ linguistic proficiency, knowledge of the target language, and communicative competence through both formative and summative practices. In the field of Computer-Assisted Language Learning (CALL), assessment has evolved to include digital tools that support feedback, adaptive testing, and performance analytics, contributing to more dynamic and personalized learning environments (Chapelle, 2006). The integration of artificial intelligence (AI) technologies has further transformed language assessment in CALL, and has a rich history (see Voss, 2024, for an overview). Tools such as Automated Writing Evaluation (AWE), Automated Essay Scoring (AES), Automatic Speech Recognition (ASR), speech analysis systems, automatic pronunciation evaluation technologies, Automated Question generation and scoring, adaptive learning platforms and so on are used to assess various language skills. These tools offer enhanced scalability, efficiency, and personalization, and are expanding possibilities for formative classroom assessment, large-scale summative evaluation, and self evaluation and/or assessment. They also enable hybrid forms of assessment that combine human judgment with machine-generated feedback, fostering teacher and AI collaboration in pedagogically meaningful ways.

Despite these technological advancements, critical questions remain regarding the pedagogical, ethical, and social implications of AI-enhanced assessment (Pack & Maloney, 2024). Questions arise about how these tools align with principles of validity, fairness, transparency, and learner-centered pedagogy. Concerns also relate to how AI-supported assessment affects the roles of both, teachers and learners, the design of instructional tasks, and the development of learner agency and AI literacy (Hauck, 2025; Stockwell, 2024). Furthermore, hybrid or human-in-the-loop approaches present opportunities and challenges for achieving equitable outcomes for diverse student populations and underrepresented languages. As AI tools shape what and how we assess, including skills beyond writing and speaking such as listening, reading, and integrated language abilities, there is a need to examine how assessment practices influence educational priorities, classroom interaction, and learner identities. AI-generated feedback, adaptive testing, and automated task creation raise questions not only about reliability and efficiency but also about the assumptions and constructs that underlie these systems.

This special issue aims to reframe and critically examine AI-enhanced language assessment within CALL contexts, with particular attention to how AI tools reshape the purposes, practices, and outcomes of assessment. It encourages interdisciplinary research that draws from applied linguistics, educational technology, and ethics to advance responsible, inclusive, and pedagogically meaningful applications of AI in language assessment. We invite empirical studies, theoretical contributions, and design-based research. Manuscripts should go beyond simple comparisons of human and AI-based assessment, and seek to push a way forward with empirical-based research that can inform assessment practices. Topics of interest include but are not limited to the following:

• Human and AI collaboration in formative assessment and feedback
• AI-enhanced diagnostic and adaptive assessment tools
• Impacts of AI-mediated assessment on teaching practices, learner agency, and AI literacy
• Hybrid assessment models that integrate teacher and machine feedback
• Validity, reliability, and transparency in AI-supported assessments
• Assessment of language skills using AI tools
• Ethical and fairness considerations in AI-mediated assessment for diverse learner populations and underrepresented languages
• AI literacy development and teacher professional learning in assessment contexts
• Responsible innovation and critical perspectives on AI in CALL-based assessment
• The impact of machine-generated feedback/assessment on learners’ metacognitive skill development
• How AI-based assessment tools address or overlook the social-interactional nature of language learning
• Authority and trustworthiness when AI and human assessments diverge significantly
• The role and legitimacy of large language models (e.g., ChatGPT) as assessment agents

References

Chapelle, C. A. (2001). Computer applications in second language acquisition. Cambridge University Press. https://doi.org/10.1017/CBO9781139524681

Hauck, M. (2025). Critical digital literacy. In L. McCallum & D. Tafazoli (Eds.), The Palgrave encyclopedia of computer-assisted language learning. Palgrave Macmillan. https://doi.org/10.1007/978-3-031-51447-0_246-1

Pack, A., & Maloney, J. (2024). Using artificial intelligence in TESOL: some ethical and pedagogical consideration. TESOL Quarterly, 58(2), 1007–1018. https://doi.org/10.1002/tesq.3320

Stockwell, G. (2024). ChatGPT in language teaching and learning: Exploring the road we’re travelling. Technology in Language Teaching & Learning, 6(1), 2273. https://doi.org/10.29140/tltl.v6n1.2273

Voss, E. (2024). Language assessment and artificial intelligence. In A. J. Kunnan (Ed.), The concise companion to language assessment (pp. 112–125). John Wiley & Sons.