Randomized evaluation of an experiential science program

By Susan Davis, Johns Hopkins University

A recent article published in AERJ by Schneider and Bradford reported the results of a cluster randomized control trial evaluating the effectiveness of the Multiple Literacies in Project-Based Learning (ML-PBL) science intervention for third graders. This study was undertaken to add to the developing evidence of the program, which had previously  undergone teaching experiments, a pilot test, and a field test. ML-PBL consists of four units, and uses a “driving question” for each lesson to spark students’ interest and engagement. It incorporates cooperative, experiential learning for the students, assessments to ensure students meet learning expectations, and teacher professional development.

The study included  2371 third graders (1165 in the experimental group,  1206 in the control group which received their business as usual science instruction) from 46 schools (23 E, 23 C) with 91 teachers (41 E, 50 C) from 111 classrooms (54 E, 57 C) during the 2018-19 school year. There were no differences between the two groups at baseline regarding ethnicity, socioeconomic status, third grade enrollment, and math/reading scores on the prior year’s state standardized testing. The ML-PBL intervention included teacher training to support classroom discussions that encouraged students to connect the content to their own lives and fostered student participation.

Post-testing was done using a test based on the MDE state test, as well as a test of social-emotional factors related to science learning. Academic results showed statistically significant gains in science for the experimental students, with an effect size of +0.27. These gains were evident regardless of gender, race, ethnicity, SES, and geographic region within the state. In addition, experimental students reported greater self-reflection and collaborative behaviors than controls. The implications of these findings for classroom practice are discussed.