A HYBRID ANALYTICAL FRAMEWORK FOR EVALUATING AI-GENERATED FEEDBACK IN ENGLISH AS SECOND LANGUAGE WRITING

Abstract

The recent launch and escalation of generative Artificial Intelligence (AI) like the one called ChatGPT have revolutionized English as a Second Language (ESL) writing instruction to create automated feedback systems that offer linguistic and structural assistance at a moment’s notice. Prior studies are still fragmented in that they tend to examine individual components like grammatical error, perceptions and/or usability aspects of feedback to writing with AI in isolation rather than providing an integrated evaluative approach. This conceptualization has shaped a gap in the methodology for comprehending the phenomenon of AI- feedback action in the several aspects of the quality of writing and pedagogy. To tackle this issue, the aim of this study is to design a multidimensional Hybrid AI Feedback Evaluation Model (HAFEM) for systematically assessing the AI feedback for ESL writing. Maintaining the linguistic accuracy, discourse quality, pedagogical value, AI reliability, and ethical integrity are five interconnected layers identified in the model reflecting Error Analysis Theory (EAT), Corpus Linguistics (Corkus), Discourse Theory, Sociocultural Learning Theory (SCT), natural language processing evaluation approaches (NLPEA), and ethical frameworks in AI (AI Ethics). The study states that considering it just a single dimension is not enough to understand the complexity of feedbacks that are produced by AI in education. According to implications, the proposed framework is of real benefit to ESL teachers, curriculum designers, practitioners in the field of artificial intelligence (AI) in language education, and policymakers by providing a structured tool to assess the efficacy, linguistic quality and ethical considerations of AI-enhanced writing systems. It also helps promote the proper and responsible use of AI in the language learning process to be more transparent. Generally, the model has implications for computational linguistics and applied linguistics, through its emphasis on interdisciplinary evaluation criteria in the context of AI-generated language feedback. The study further highlights the needs to evaluate AI-generated feedback for ESL writing from a more comprehensive perspective, beyond the fragmented one, and from a more theoretical one. By incorporating linguistic, discourse, pedagogical, technical, and ethical aspects in a single analytical framework, the HAFEM model overcomes the lack of a comprehensive modeling approach and paves the way for future interdisciplinary studies on the use of AI in L2 learning methodologies.