For researchers seeking to improve education, a common goal is to identify teaching practices that have causal benefits in classroom settings. To test whether an instructional practice exerts a causal influence on an outcome measure, the most straightforward and compelling method is to conduct an experiment. While experimentation is common in…

Findings from experimental studies have increasingly been used to inform policy in school settings. Thus far, the populations in many of these studies are typically defined in a cross-sectional context; namely, the populations are defined in the same academic year in which the study took place or the population is defined at a fixed time point.…

Randomized controlled trials in education are necessary to keep pace with the evidence-based practices demanded by schools and the nation. In this study, kindergarten students in a school district in Indiana were randomized on the class level: The experimental condition consisted of students utilizing a computer-adaptive reading program for 15…

Disruptive behavior problems frequently emerge in the preschool years and are associated with numerous, long-term negative outcomes, including comorbid disorders. First Step is a psychosocial early intervention with substantial empirical evidence supporting its efficacy among young children (Walker et al., 2014). The present study reports on a…

Policymakers are increasingly interested in the extent to which experimental results generalize from a sample to a population of inference. When the sample is not randomly selected, propensity score methods are used to reweight the sample. Subclassification by propensity score is commonly used in which the population is partitioned into strata…

We explore the conditions under which short, comparative interrupted time-series (CITS) designs represent valid alternatives to randomized experiments in educational evaluations. To do so, we conduct three within-study comparisons, each of which uses a unique data set to test the validity of the CITS design by comparing its causal estimates to…

This five-year study focused on school processes that promoted the scaling-up of a high school academic literacy framework, Reading Apprenticeship, developed by WestEd's Strategic Literacy Initiative (SLI). Implementing an innovative strategy for scaling-up involving school-based cross-disciplinary teacher teams, SLI brought the framework to 274…

As the first major effort to use a behavioral economics lens to examine human services programs that serve poor and vulnerable families in the United States, the Behavioral Interventions to Advance Self-Sufficiency (BIAS) project demonstrated the value of applying behavioral insights to improve the efficacy of human services programs. The BIAS…

Recent methods to improve generalizations from nonrandom samples typically invoke assumptions such as the strong ignorability of sample selection, which is challenging to meet in practice. Although researchers acknowledge the difficulty in meeting this assumption, point estimates are still provided and used without considering alternative…

Field experiments in education frequently assign entire groups such as schools to treatment or control conditions. These experiments incorporate sometimes a longitudinal component where for example students are followed over time to assess differences in the average rate of linear change, or rate of acceleration. In this study, we provide methods…

The Behavioral Interventions to Advance Self-Sufficiency (BIAS) project conducted 15 randomized controlled trials of behavioral interventions across eight states, in the domains of work support, child support, and child care. BIAS used a systematic approach called "behavioral diagnosis and design" to develop the interventions and their…

This study evaluated the efficacy of the Preschool First Step (PFS) to Success early intervention for children at risk for attention deficit hyperactivity disorder (ADHD). PFS is a targeted intervention for children 3-5 years old with externalizing behavior problems and addresses secondary prevention goals and objectives. As part of a larger…

The study authors examined the effects of using "Diagnostic Assessment Tools" ("DAT") on mathematics and reading outcomes for students in Indiana schools during the 2009-10 academic year. "DAT" consists of interim assessment tools--Wireless Generation's mCLASS for students in grades K-2 and CTB/ McGraw-Hill's Acuity…

Policy-makers are frequently interested in understanding how effective a particular intervention may be for a specific (and often broad) population. In many fields, particularly education and social welfare, the ideal form of these evaluations is a large-scale randomized experiment. Recent research has highlighted that sites in these large-scale…

The goal of this paper is to provide guidance for applied education researchers in using multi-level data to study the effects of interventions implemented at the school level. Two primary approaches are currently employed in observational studies of the effect of school-level interventions. One approach employs intact school matching: matching…