Background: Multilevel randomized controlled trials (MRCTs) have been widely used to evaluate the causal effects of educational interventions. Traditionally, educational researchers and policymakers focused on the average treatment effects (ATE) of the intervention. Recently there has been an increasing interest in evaluating the heterogeneity of…

Many public and private initiatives have focused on improving prekindergarten (PK) through Grade 12 mathematics education, given that the mathematics content learned before college is foundational for later learning and college success across STEM subjects. Although decades of educational research have studied PK-12 mathematics interventions, the…

The goal of this study is to better understand how methods for estimating treatment effects of latent groups operate. In particular, the authors identify where violations of assumptions can lead to biased estimates, and explore how covariates can be critical in the estimation process. For each set of approaches, the authors first review the…

Across the disciplines of economics, political science, public policy, and now, education, the randomized controlled trial (RCT) is the preferred methodology for establishing causal inference about program impacts. But randomized experiments are not always feasible because of ethical, political, and/or practical considerations, so non-experimental…

Despite recent emphasis on the use of randomized control trials (RCTs) for evaluating education interventions, in most areas of education research, observational methods remain the dominant approach for assessing program effects. Over the last three decades, the within-study comparison (WSC) design has emerged as a method for evaluating the…

This paper examines the "ecological validity" of a randomized controlled trial (RCT) conducted in 42 high schools to measure the impact of a content literacy program. This study of comparable schools implementing the same program in different contexts highlights characteristics that are often not attended to in rigorous effectiveness…

Randomized experiments have long been established as the gold standard for addressing causal questions. However, experiments are not always feasible or desired, so observational methods are also needed. When multiple observations on the same variable are available, a repeated measures design may be used to assess whether a treatment administered…

The purpose of this study is to study the performance of different methods (inverse probability weighting and estimation of informative bounds) to control for differential attrition by comparing the results of different methods using two datasets: an original dataset from Portland Public Schools (PPS) subject to high rates of differential…

American Institutes for Research (AIR) is conducting an impact evaluation of The United Nations Children's Emergency Fund's (UNICEF's) teacher-training program and reinforcing text messages that aim to provide meaningful knowledge regarding the transformative potential of positive gender socialization in education for peace building in the region…

Researchers are increasingly using comparative interrupted time series (CITS) designs to estimate the effects of programs and policies when randomized controlled trials are not feasible. In a simple interrupted time series design, researchers compare the pre-treatment values of a treatment group time series to post-treatment values in order to…

Policy-makers are frequently interested in understanding how effective a particular intervention may be for a specific (and often broad) population. In many fields, particularly education and social welfare, the ideal form of these evaluations is a large-scale randomized experiment. Recent research has highlighted that sites in these large-scale…

Children from low-income and ethnic minority backgrounds have demonstrated substantially lower levels of math achievement than their middle class majority peers for decades. The present study addressed two research questions: (1) when used as a supplement to typical classroom instruction and in isolation from the larger curriculum, does Building…

The SUN approach to biological energy transfer education is fundamentally different from past practices that trace chemical and energy inputs and outputs. The SUN approach uses a hydrogen fuel cell to convince learners that electrons can move from one substance to another based on differential attraction. With a hydrogen fuel cell, learners can…

The goal of this paper is to provide guidance for applied education researchers in using multi-level data to study the effects of interventions implemented at the school level. Two primary approaches are currently employed in observational studies of the effect of school-level interventions. One approach employs intact school matching: matching…

This randomized control study compares the efficacy of two response-to-intervention (RTI) models: (1) Dynamic RTI, which immediately refers grade 1 students with the weakest skills to the most intensive intervention supports (Tier 2 or Tier 3); and (2) Typical RTI, which starts all students in Tier 1 and after 8 weeks, decides whether students who…