A Network Connecting School Leaders From Around The Globe
Communicating accurate and informative research results to teachers
By Justin Hill, Johns Hopkins University
Educational research findings on effective teaching practices must be accurately communicated to teachers to impact classroom teaching. However, little research exists to indicate how teachers perceive different effect size metrics commonly presented to educators. Lortie-Forgues and colleagues recently conducted two studies to examine teacher perceptions of informativeness and effectiveness when presented with information using different metrics. They tested five effect size metrics commonly presented in literature directed toward teachers: Months of Progress, Percentile Gain, Cohen’s U3, Threshold (the additional proportion of students attaining a specific threshold mark), and Test Scores.
The findings suggest that researchers can make their research more engaging for teachers by using certain metrics, but that they also must use caution to ensure that effectiveness is accurately communicated. In the first study, teachers found Threshold metrics to be significantly more informative than Test Score (ES = +0.30; p ≤ .01), Percentile Gain (ES = +1.25; p ≤ .001), and Cohen’s U3 (ES = +1.31; p ≤ .001). There was no significant difference found between Threshold and Months of Progress. In the second study, teachers were presented with different effect size metrics and asked to determine which effect was largest. Unknown to the teachers, all the effect size metrics indicated identical effects. Teachers perceived effects communicated in terms of Months of Progress as significantly more effective (p ≤ .001) than those communicated in terms of Threshold (ES = +0.40), Cohen’s U3 (ES = +0.58), Percentile Gain (ES = +0.68) and Test Score (ES = +1.32). These findings suggest that teachers find the Months of the Progress and Threshold metrics to be most informative and engaging, but these same metrics are also most likely to convey a sense of the effect sizes being larger than the research indicates. The authors suggest that researchers may want to communicate effect sizes using multiple metrics as a method of both engaging teachers in the findings while also helping to ensure that accurate information is being communicated.