As a result of the evidence-based decision-making movement, the number of randomized trials evaluating educational programs and curricula has increased dramatically over the past 20 years. Policy makers and practitioners are encouraged to use the results of these trials to inform their decision making in schools and school districts. At the same…

To plan cluster-randomized trials with sufficient statistical power to detect intervention effects on student achievement, researchers need multilevel design parameters, including measures of between-classroom and between-school differences and the amounts of variance explained by covariates at the student, classroom, and school level. Previous…

Educational stakeholders are keen to know the magnitude and importance of different interventions. However, the way evidence is communicated to support understanding of the effectiveness of an intervention is controversial. Typically studies in education have used the standardised mean difference as a measure of the impact of interventions. This…

With the increasing demand for evidence-based research on teacher effectiveness and improving student achievement, more impact studies are being conducted to examine the effectiveness of professional development (PD) interventions. Cluster randomized trials (CRTs) are often carried out to assess PD interventions that aim to improve both teacher…

To plan group-randomized trials where treatment conditions are assigned to schools, researchers need design parameters that provide information about between-school differences in outcomes as well as the amount of variance that can be explained by covariates at the student (L1) and school (L2) levels. Most previous research has offered these…

Recent methods to improve generalizations from nonrandom samples typically invoke assumptions such as the strong ignorability of sample selection, which is challenging to meet in practice. Although researchers acknowledge the difficulty in meeting this assumption, point estimates are still provided and used without considering alternative…

While the adjusted cohort graduation rate of students in the United States has reached 80 percent during the 2011-12 school year (Stetser & Stillwell, 2014), in the School District of Philadelphia (SDP) only 57% of its students graduate from high school in four years, which is typical of large urban school districts (Swanson, 2009). Moreover,…

The purpose of this paper is to showcase new research that seeks to provide guidance on the heterogeneity of treatment effects by utilizing the variance of demographic differences in state assessments. This study is focused on a simple randomized block design where students are nested within schools, and within each school students are randomized…

Research has shown that teacher effectiveness is the most important school-based factor that influences student outcomes, including student achievement. Studies have also shown that there is substantial variation in teacher effectiveness, such that some teachers improve student outcomes at a much faster rate than others. The strong influence of…

Based on the experience of evaluating 2 cross-age peer-tutoring interventions, we argue that researchers need to pay greater attention to causal mechanisms within the context of school-based randomised controlled trials. Without studying mechanisms, researchers are less able to explain the underlying causal processes that give rise to results from…

While performance-based compensation systems in primary and secondary education are no longer a novelty in the US, they are still not yet common in the UK. Research from the US finds mixed results regarding the effects of performance-related pay on pupil achievement, but there is limited evidence around this concept from the UK. Therefore, RAND…

The objective of this study was to look across strategies used in a wide range of studies to build a framework for researchers to use in conceptualizing the recruitment process. This paper harvests lessons learned across 19 randomized controlled trials in K-12 school settings conducted by a leading research organization to identify strategies that…

Background: Educational interventions are often complex, and their outcomes could be due to factors not focused on in the impact evaluation. Therefore, educational evaluations using a randomised control trial (RCT) design approach need to go beyond obtaining the impact results alone. Purpose: Process evaluation is embedded in the evaluation design…

Despite widespread recognition among policy makers, educational leaders, and the nation as a whole of the importance of science, engineering, and technology as drivers of the future of the country and society, the science achievement of America's students is mediocre at best, in comparison to that of international peers. On the 2012 PISA tests in…

One purpose of education research is to develop and rigorously evaluate the effectiveness of programs for supporting students' learning and achievement. The Institute of Education Sciences has amplified that purpose (Shadish & Cook, 2009) and attempted to improve the methodological standards for conducting such work--primarily through the What…