The present article considers a fundamental question in evaluation research: "By how much do program effects vary across sites?" The article first presents a theoretical model of cross-site impact variation and a related estimation model with a random treatment coefficient and fixed site-specific intercepts. This approach eliminates…

Student academic self-assessment engages the students in deliberate reflection about what they are learning and how they are learning it. This intervention study investigated the effects of self-assessment training on students' perceptions towards self-assessment in two Singaporean primary schools. The study, which used a pretest-posttest design,…

We report on a practitioner implementation of Fusion, a first-grade mathematics intervention. Studies such as this evaluation of a loose implementation under realistic conditions are important to curriculum developers' understanding of how evidence-based programs and tools work under a variety of implementation scenarios. In this…

We report on a practitioner implementation of Fusion, a first grade mathematics intervention. Studies such as this evaluation of a loose implementation under realistic conditions are important to curriculum developers' understanding of how evidence-based programs and tools work under a variety of implementation scenarios. In this…

We report on an effectiveness evaluation of the Youth Empowerment Solutions (YES) program. YES applies empowerment theory to an after-school program for middle school students. YES is an active learning curriculum designed to help youth gain confidence in themselves, think critically about their community, and work with adults to create positive…

This study evaluated two 20-week reading interventions for pupils entering secondary school with reading difficulties. The interventions were delivered by trained teaching assistants (three 35-min sessions per week). 287 pupils (ages 11-13) from 27 schools were randomly allocated to three groups: reading intervention (targeting word recognition…

We investigated methods of including covariates in two-level models for cluster randomized trials to increase power to detect the treatment effect. We compared multilevel models that included either an observed cluster mean or a latent cluster mean as a covariate, as well as the effect of including Level 1 deviation scores in the model. A Monte…

Engineers and policymakers have expressed concern that too few students enter the engineering pipeline. This has led to many efforts to engage students in engineering in after-school programs, summer programs, and more recently, in school curricula. The expectation is that, through these efforts, greater numbers of more demographically diverse…

The present study examined parent-teacher congruent communication within conjoint behavioral consultation (CBC). Specifically, the study purpose was to determine the extent to which congruence in parent-teacher communication (i.e., the degree to which parents and teachers view their communication in a similar fashion) moderated CBC's effects on…

Using a randomized controlled trial, we tested a new teacher professional development program for increasing the language and literacy skills of young Latino English learners with 45 teachers and 105 students in 12 elementary schools. School-based teams randomly assigned to the intervention received professional development focused on cultural…

This study evaluated the results of a social and emotional learning (SEL) program on academic achievement among students attending a large, urban, high-risk school district. Using a cluster-randomized design, 24 elementary schools were assigned to receive either the intervention curriculum ("Promoting Alternative Thinking Strategies", or…

A multilevel latent transition analysis (LTA) with a mixture IRT measurement model (MixIRTM) is described for investigating the effectiveness of an intervention. The addition of a MixIRTM to the multilevel LTA permits consideration of both potential heterogeneity in students' response to instructional intervention as well as a methodology for…

In education randomized control trials (RCTs), the misreporting of student outcome data could lead to biased estimates of average treatment effects (ATEs) and their standard errors. This article discusses a statistical model that adjusts for misreported binary outcomes for two-level, school-based RCTs, where it is assumed that misreporting could…

In randomized control trials (RCTs) of educational interventions, there is a growing literature on impact estimation methods to adjust for missing student outcome data using such methods as multiple imputation, the construction of nonresponse weights, casewise deletion, and maximum likelihood methods (see, for example, Allison, 2002; Graham, 2009;…

This paper examines the estimation of two-stage clustered RCT designs in education research using the Neyman causal inference framework that underlies experiments. The key distinction between the considered causal models is whether potential treatment and control group outcomes are considered to be fixed for the study population (the…