I. Purpose of the Study: Detecting whether interventions work or not (through main effect analysis) can provide empirical evidence regarding the causal linkage between malleable factors (e.g., interventions) and learner outcomes. In complement, moderation analyses help delineate for whom and under what conditions intervention effects are most…

Background: Randomized controlled trials (RCTs), or experiments, are the gold standard for intervention evaluation. However, the main appeal of RCTs--the clean identification of causal effects--can be compromised by interference, when one subject's actions can influence another subject's behavior or outcomes. In this paper, we formalize and study…

Background: Education scientists are increasingly interested in constructing interventions that are adaptive over time to suit the evolving needs of students, classrooms, or schools. Such "adaptive interventions" (also referred to as dynamic treatment regimens or dynamic instructional regimes) determine which treatment should be offered…

Random encouragement designs are randomized controlled trials (RCTs) that test interventions aimed at increasing participation in a program or activity whose take up is not universal. In these RCTs, instead of randomizing individuals or clusters directly into treatment and control groups to participate in a program or activity, the randomization…

Context: Assessments of baseline equivalency of intervention and control groups, "balance," play a critical role in evaluating educational interventions. The highest What Works Clearinghouse (WWC) of the Institute of Educational Studies (IES) standard for educational studies, "Meets WWC Design Standards Without Reservations,"…

Background: Efficacy trials in education often possess a motivating theory of change: how and why should the desired improvement in outcomes occur as a consequence of the intervention? In scenarios with repeated measurements, certain subgroups may be more or less likely to manifest a treatment effect; the theory of change (TOC) provides guidance…

Evaluators of education interventions increasingly need to design studies to detect impacts much smaller than the 0.20 standard deviations that Cohen (1988) characterized as "small." For example, an evaluation of Response to Intervention from the Institute of Education Sciences (IES) detected impacts ranging from 0.13 to 0.17 standard…

Background/Context: When a researcher tests an educational program, product, or policy in a randomized controlled trial (RCT) and detects a significant effect on an outcome, the intervention is usually classified as something that "works." When the expected effects are not found, however, there is seldom an orderly and transparent…

Background: It has become increasingly important to shift education research and programming from a deficit focused perspective to a positive youth development perspective in order to identify and promote factors that support students' positive functioning and outcomes. One outgrowth of positive youth development is Positive Mental Health (PMH),…

Many public and private initiatives have focused on improving prekindergarten (PK) through Grade 12 mathematics education, given that the mathematics content learned before college is foundational for later learning and college success across STEM subjects. Although decades of educational research have studied PK-12 mathematics interventions, the…

Background: In fall 2020, community colleges (CCs) served nearly five million students, representing 29% of U.S. undergraduates. Despite providing unprecedented access to postsecondary education, rates of degree attainment are low. Among first-time, full-time, degree/certificate-seeking students whose first postsecondary school is a CC, only 31%…

Several studies suggest that values-affirmation can serve as a simple, yet powerful, tool for dramatically reducing achievement gaps. Because subtle variations in implementation procedures may explain some of the variation in these findings, it is crucial for researchers to measure the fidelity with which interventions are implemented. The authors…

A common design in education research for interventions operating at a group or cluster level is a cluster randomized trial (CRT) (Bloom, 2005). In CRTs, intact clusters (e.g., schools) are assigned to treatment conditions rather than individuals (e.g., students) and are frequently an effective way to study interventions because they permit…

Randomized controlled trials (RCTs) are increasingly used to evaluate programs and interventions in order to inform education policy and practice. High quality reports of these RCTs are needed for interested readers to understand the rigor of the study, the interventions tested, and the context in which the evaluation took place (Mayo-Wilson et…

This paper examines the "ecological validity" of a randomized controlled trial (RCT) conducted in 42 high schools to measure the impact of a content literacy program. This study of comparable schools implementing the same program in different contexts highlights characteristics that are often not attended to in rigorous effectiveness…