Comparing the cost-effectiveness of algebraic technological apps for seventh graders

By Nathan Storey, Johns Hopkins University

Despite increasing distance from the main brunt of the COVID-19 pandemic, students in the United States have continued to struggle academically, particularly in mathematics. As educators and school leaders seek to address these challenges, they are confronted with the dual considerations of assessing the academic impact and managing the financial costs associated with implementing new programs. Consequently, studies on the cost-effectiveness of educational interventions have become invaluable resources for decision-making.

In a recent study, Finster and colleagues build on existing research of the impact of game-based algebraic technology applications, as originally described in the work of Decker-Woodrow and colleagues. Specifically, Finster and colleagues examined the cost-effectiveness of three algebraic technological applications for seventh grade students: From Here to There (FH2T), Dragon Box 12+ (DragonBox, previously reviewed in BEiB here), and Immediate Feedback. FH2T engages students with math concepts through a discovery-based approach, emphasizing puzzles over procedural steps, with the aim of enhancing engagement, efficacy, and interest.  DragonBox is designed to teach algebraic concepts also through a discovery puzzle-based approach, as well as offering different challenge levels and adaptability. Lastly,  Immediate Feedback is a free, online tutoring system, and features formative assessment and timely feedback rather than game-based learning.

The study compared these three applications to an active control group that replicated traditional homework assignments using technology. The final sample included data from nine schools, 34 teachers, 127 classes, and a total of 1,850 students (comprising 753 for FH2T, 350 for DragonBox, 381 for Immediate Feedback, and 366 for the Active Control group). The researchers employed hierarchical linear modeling (HLM) to assess the effects of the interventions, while the ingredients method was utilized to estimate their costs.

The analysis identified both FH2T and DragonBox as low-cost interventions that can efficiently improve algebraic performance for students. Cost-effectiveness ratios were calculated by dividing the average cost per participant by the average effectiveness for each application. In this case, DragonBox was found to cost $55 per student, with a +0.26 average ES (cost-effectiveness ratio of $206), while FH2T cost $39 per student, with an +0.14 average ES (cost-effectiveness ratio of $291). These ratios are representative of implementing interventions in person, as designed and practiced in this study.