Is Literacy First effective in the long term? Insights from a multisite RCT

Carmen Pannone, University of Cagliari, Italy

The effectiveness of early literacy tutoring interventions in the short term, specifically in grades K-2, has been supported by various studies. One such intervention is Literacy First, which offers tutoring to students from kindergarten through Grade 2 in underserved communities in the central Texas area, especially those reading below grade level at the beginning of the school year. Participating students receive daily one-on-one 30-minute lessons from a highly trained volunteer, customized based on data obtained through a response-to-intervention approach.

The American Institute of Research (AIR) has been investigating whether Literacy First, when implemented in Grade 2, exhibits persistent effects up to three years after program completion. The evaluation, conducted through a multisite randomized control trial, involved 22 elementary schools within a Texan school district. Following a screening process that utilized standardized assessments, the randomization of the 1,046 Grade 2 students who had scored at Tier 2 and were eligible to participate in the study resulted in a sample of 798 students (525 from Cohort 1 and 273 from Cohort 2). Using a two-level hierarchical linear model to analyze the data, the study examined the impact of Literacy First as measured by the STAAR Grades 4 and 5 reading, the STAAR Grade 4 writing, and the STAAR Grade 5 science.

The results showed a positive effect of the intervention on the STAAR Grade 4 reading assessment (ES = +0.12), the STAAR Grade 4 writing assessment (ES = +0.04), and the STAAR Grade 5 reading assessment (ES = +0.04). Conversely, students who received Literacy First tutoring achieved lower results in the science assessment compared to the control group (ES = -0.08). None of these observed differences reached statistical significance, possibly due, in part, to the impact of the COVID-19 pandemic, as STAAR Grade 4 results were only available for Cohort 1, and both cohorts were affected by the pandemic between the time they received the tutoring and when they completed the analyzed assessments. Taking the impact of the pandemic into account, these results, combined with previously conducted studies, suggest that the effects of Literacy First may be most pronounced shortly after completing the program, with a diminishing impact over time.