Proceedings of the 9th International Conference on Advanced Research in Teaching and Education
Year: 2024
DOI:
[PDF]
Leveraging Generative AI for University-Level English Proficiency: Empirical Findings on Automated Feedback and Learning Outcomes
Hsuehi Lo
ABSTRACT:
This study explores how feedback generated by large language models (LLMs) affects the essay-writing skills of university students in Hong Kong. It examines the potential of generative AI to enhance student essay revisions, its influence on engagement with writing tasks, and the emotions students experience during the revision process. Conducted as a randomized controlled trial, the research compares the performance and experiences of 918 language students at a Hong Kong university, where some students received AI-generated feedback (via GPT-3.5-turbo LLM) and others did not. The impact of AI-generated feedback was evaluated using both objective and subjective measures, including statistical analysis of essay grades, student surveys capturing motivation and emotional responses, and thematic analysis of interviews. The results showed significant improvements in the quality of essays for those who received AI feedback. Quantitative data indicated strong effect sizes with statistical significance, while qualitative feedback underscored increased engagement, motivation, and a range of emotional responses during the revision process.
keywords: AI in education, Automated feedback, Emotional responses in learning, Essay revisions, Writing proficiency