For researchers seeking to improve education, a common goal is to identify teaching practices that have causal benefits in classroom settings. To test whether an instructional practice exerts a causal influence on an outcome measure, the most straightforward and compelling method is to conduct an experiment. While experimentation is common in…

Findings from experimental studies have increasingly been used to inform policy in school settings. Thus far, the populations in many of these studies are typically defined in a cross-sectional context; namely, the populations are defined in the same academic year in which the study took place or the population is defined at a fixed time point.…

Randomized controlled trials in education are necessary to keep pace with the evidence-based practices demanded by schools and the nation. In this study, kindergarten students in a school district in Indiana were randomized on the class level: The experimental condition consisted of students utilizing a computer-adaptive reading program for 15…

Disruptive behavior problems frequently emerge in the preschool years and are associated with numerous, long-term negative outcomes, including comorbid disorders. First Step is a psychosocial early intervention with substantial empirical evidence supporting its efficacy among young children (Walker et al., 2014). The present study reports on a…

Policymakers are increasingly interested in the extent to which experimental results generalize from a sample to a population of inference. When the sample is not randomly selected, propensity score methods are used to reweight the sample. Subclassification by propensity score is commonly used in which the population is partitioned into strata…

We explore the conditions under which short, comparative interrupted time-series (CITS) designs represent valid alternatives to randomized experiments in educational evaluations. To do so, we conduct three within-study comparisons, each of which uses a unique data set to test the validity of the CITS design by comparing its causal estimates to…

This five-year study focused on school processes that promoted the scaling-up of a high school academic literacy framework, Reading Apprenticeship, developed by WestEd's Strategic Literacy Initiative (SLI). Implementing an innovative strategy for scaling-up involving school-based cross-disciplinary teacher teams, SLI brought the framework to 274…

Recent methods to improve generalizations from nonrandom samples typically invoke assumptions such as the strong ignorability of sample selection, which is challenging to meet in practice. Although researchers acknowledge the difficulty in meeting this assumption, point estimates are still provided and used without considering alternative…

This study evaluated the efficacy of the Preschool First Step (PFS) to Success early intervention for children at risk for attention deficit hyperactivity disorder (ADHD). PFS is a targeted intervention for children 3-5 years old with externalizing behavior problems and addresses secondary prevention goals and objectives. As part of a larger…

Policy-makers are frequently interested in understanding how effective a particular intervention may be for a specific (and often broad) population. In many fields, particularly education and social welfare, the ideal form of these evaluations is a large-scale randomized experiment. Recent research has highlighted that sites in these large-scale…

The goal of this paper is to provide guidance for applied education researchers in using multi-level data to study the effects of interventions implemented at the school level. Two primary approaches are currently employed in observational studies of the effect of school-level interventions. One approach employs intact school matching: matching…

The nested data structure inherent in education (i.e. students nested in schools nested in districts) makes intact school matching an appealing approach in observational studies of educational interventions and policies for both theoretical and practical purposes. This paper provides guidance to applied education researchers who are employing…

We report on the scaling up of a high school content literacy framework, Reading Apprenticeship, over a period of four years as part of the independent evaluation of an Investing in Innovation grant from the U.S. Department of Education to WestEd's Strategic Literacy Institute (SLI). Our goal was to understand the school processes that support…

Preschool children with Autism Spectrum Disorder (ASD) may not always be recognized as such during their early years, but some of their behavioral problems may nonetheless prompt a referral for behavioral intervention. Whether such an intervention brings any benefit has not been well studied. We identified a subsample of 34 preschool children at…

Preschool children with Autism Spectrum Disorder (ASD) may not always be recognized as such during their early years, but some of their behavioral problems may nonetheless prompt a referral for behavioral intervention. Whether such an intervention brings any benefit has not been well studied. We identified a subsample of 34 preschool children at…