Assessing the quality of feedback from humans and ChatGPT on students’ writing

Assessing the quality of feedback from humans and ChatGPT on students’ writing

By Feifei Wang, The Chinese University of Hong Kong

 

Automated writing evaluation (AWE) tools aid educators by quickly evaluating student writing, but often at the expense of accuracy and clarity, and setting them up requires substantial effort and resources. In contrast, generative AI like ChatGPT provides timely, targeted, and adaptive feedback that can enhance student writing. A recent study compared the quality of feedback from ChatGPT and human raters, with potential implications for process-based writing evaluation.

 

The study involved 200 students randomly sampled across grades 6-12 from two school districts in Southern California. Over two 50-minute class periods, students were tasked with responding to one of two writing prompts. Each prompt demanded that they read four primary and secondary sources and craft an argument analyzing historical interpretations using evidence, reasoning, and required writing skills. Sixteen experienced secondary educators from various disciplines were recruited and trained to provide formative feedback. This feedback was then compared to ChatGPT’s feedback, which was prompted to reflect the same context used by the educators. The researchers coded and analyzed the feedback for comparison.

 

The findings revealed  that well-trained, paid, and relatively time-rich human raters offered higher quality feedback in four of five key areas: clarity of improvement directions, accuracy, prioritization of essential features, and use of a supportive tone (the fifth area: criteria-based feedback). However,  ChatGPT’s writing feedback closely matched  human feedbac..., and the differences in quality between ChatGPT and human feedback  were modest, considering both overall quality and time savings. The researchers concluded that generative AI could be a useful tool in specific situations, especially for formative drafts or when highly-trained educators are unavailable.

Views: 3

Reply to This

JOIN SL 2.0

SUBSCRIBE TO

SCHOOL LEADERSHIP 2.0

School Leadership 2.0 is the premier virtual learning community for school leaders from around the globe.  Our community is a subscription based paid service ($19.95/year or only $1.99 per month for a trial membership)  which will provide school leaders with outstanding resources. Learn more about membership to this service by clicking one our links below.

 

Click HERE to subscribe as an individual.

 

Click HERE to learn about group membership (i.e. association, leadership teams)

__________________

CREATE AN EMPLOYER PROFILE AND GET JOB ALERTS AT 

SCHOOLLEADERSHIPJOBS.COM

FOLLOW SL 2.0

© 2024   Created by William Brennan and Michael Keany   Powered by

Badges  |  Report an Issue  |  Terms of Service