In paired experiments, participants are grouped into pairs with similar characteristics, and one observation from each pair is randomly assigned to treatment. The resulting treatment and control groups should be well-balanced; however, there may still be small chance imbalances. Building on work for completely randomized experiments, we propose a…

This article assesses the likely generalizability of educational treatment-effect estimates from regression discontinuity designs (RDDs) when treatment assignment is based on academic pretest scores. Our assessment uses data on outcome and pretest measures from six educational experiments, ranging from preschool through high school, to estimate…

This article assesses the likely generalizability of educational treatment-effect estimates from regression discontinuity designs (RDDs) when treatment assignment is based on academic pretest scores. Our assessment uses data on outcome and pretest measures from six educational experiments, ranging from preschool through high school, to estimate…

Questions regarding the generalizability of results from educational experiments have been at the forefront of methods development over the past five years. This work has focused on methods for estimating the effect of an intervention in a well-defined inference population (e.g., Tipton, 2013; O'Muircheartaigh and Hedges, 2014); methods for…

Little is known empirically about intraclass correlations (ICCs) for multisite cluster randomized trial (MSCRT) designs, particularly in science education. In this study, ICCs suitable for science achievement studies using a three-level (students in schools in districts) MSCRT design that block on district are estimated and examined. Estimates of…

Recent federal legislation, such as the "No Child Left Behind Act" of 2001 and the "Education Sciences Reform Act (ESRA)" of 2002, has insisted that educational evaluations use rigorous research designs with quantitative outcome measures. In particular, the Institute of Education Sciences (IES) at the U.S. Department of…

A cluster-randomized trial (CRT) relies on random assignment of intact clusters to treatment conditions, such as classrooms or schools (Raudenbush & Bryk, 2002). One specific type of CRT, a multi-site CRT (MSCRT), is commonly employed in educational research and evaluation studies (Spybrook & Raudenbush, 2009; Spybrook, 2014; Bloom,…

The capacity of the field to conduct power analyses for group randomized trials (GRTs) of educational interventions has improved over the past decade (Authors, 2009). However, a power analysis depends on estimates of design parameters. Hence it is critical to build the empirical base of design parameters for GRTs across a variety of outcomes and…

The consequences of failing core academic courses during the first year of high school are dire. More students fail courses in ninth grade than in any other grade, and a disproportionate number of these students subsequently drop out (Herlihy, 2007). As shown in Chicago and elsewhere, academic performance in core courses during the first year of…

Algebra is considered a key gatekeeper for higher-level mathematics course-taking in high school and for college enrollment (Adelman, 2006; Gamoran & Hannigan, 2000). Yet, algebra pass rates are consistently low in many places (Higgins, 2008; Ham & Walker, 1999; Helfand, 2006), including Chicago Public Schools (CPS). This is of particular…