Using ChatGPT to Assign Overall Band Scores to IELTS Writing Task 2 Essays: A comparison with Human Rater Scores

Abstract Book of the 10th International Conference on Advanced Research in Education

Year: 2026

[PDF]

Using ChatGPT to Assign Overall Band Scores to IELTS Writing Task 2 Essays: A comparison with Human Rater Scores

Dr. Anood Al Shibli

ABSTRACT:

The noticeable growth in the use of AI in language assessment has raised questions about the reliability of automated writing evaluation system compared with the teachers’ marking (Karatay & Karatay, 2024; Pack et al., 2024; Ramesh & Sanampudi, 2022). This study investigated the difference in assigning IELTS band to a writing two IELTS task. Six hundred and sixty-nine essays were used for this purpose. Each essay was assessed using the four IELTS writing criteria: Task Response, Coherence and Cohesion, Lexical Resource, and Grammatical Range and Accuracy. The findings showed that, on average, ChatGPT, assigned 5.5 overall IELTS Band while 4.5 was given to these essays by ex-IELTS examiner. In detail, 475 essays were awarded higher scores by ChatGPT than IELTS rater leaving only 89 essays with identical ratings and even 89 essays were scored less than IELTS rater. In addition, criterion-level results demonstrated that ChatGPT assigned evidently higher scores for Task Response and Coherence and Cohesion, whereas smallest difference was observed in Grammatical Range and Accuracy. The findings suggested that although ChatGPT scores are relatively associated with human ratings, ChatGPT provided evidence of a systematic tendency to assign higher bands. Therefore, it should not be used as a direct replacement for trained human raters without further careful and judgment of human oversight. This finding, however, does not deny the fact that ChatGPT can be used to focus on improving grammatical accuracy, particularly at lower level of proficiency learners.

Keywords: Artificial Intelligence, Automated Writing Evaluation System, Human Rater, IELTS