In education, multisite trials involve randomization of pupils into intervention and comparison groups within schools. Most analytical models in multisite educational trials ignore that the impact of an intervention may be school dependent. This study investigates the impact of statistical models on the uncertainty associated with an effect size…

In randomized control trials (RCT), the recent focus has shifted to how an intervention yields positive results on its intended outcome. This aligns with the recent push of implementation science in healthcare (Bauer et al., 2015) but goes beyond this. RCTs have moved to evaluating the theoretical framing of the intervention as well as differing…

Cost-effectiveness analysis is a widely used educational evaluation tool. The randomized controlled trials that aim to evaluate the cost-effectiveness of the treatment are commonly referred to as randomized cost-effectiveness trials (RCETs). This study provides methods of power analysis for two-level multisite RCETs. Power computations take…

Researchers are often interested in whether the effects of an intervention differ conditional on individual- or group-moderator variables such as children's characteristics (e.g., gender), teacher's background (e.g., years of teaching), and school's characteristics (e.g., urbanity); that is, the researchers seek to examine for whom and under what…

Current practice for conducting power analyses in hierarchical trials using survey based ICC and effect size estimates may be misestimating power because ICCs are not being adjusted to account for treatment effect heterogeneity. Results presented in Table 1 show that the necessary adjustments can be quite large or quite small. Furthermore, power…

Designs that facilitate inferences concerning both the total and indirect effects of a treatment potentially offer a more holistic description of interventions because they can complement "what works" questions with the comprehensive study of the causal connections implied by substantive theories. Mapping the sensitivity of designs to…

Cluster randomized experiments are ubiquitous in modern education research. Although a variety of modeling approaches are used to analyze these data, perhaps the most common methodology is a normal mixed effects model where some effects, such as the treatment effect, are regarded as fixed, and others, such as the effect of group random assignment…

When cluster randomized experiments are analyzed as if units were independent, test statistics for treatment effects can be anticonservative. Hedges proposed a correction for such tests by scaling them to control their Type I error rate. This article generalizes the Hedges correction from a posttest-only experimental design to more common designs…

The purpose of this paper is to showcase new research that seeks to provide guidance on the heterogeneity of treatment effects by utilizing the variance of demographic differences in state assessments. This study is focused on a simple randomized block design where students are nested within schools, and within each school students are randomized…

Randomized experiments are commonly used to evaluate the effectiveness of educational interventions. The goal of the present investigation is to develop small-sample corrections for multiple contrast hypothesis tests (i.e., F-tests) such as the omnibus test of meta-regression fit or a test for equality of three or more levels of a categorical…

This randomized, controlled trial, comparing the Comprehensive Autism Program (CAP) and business as usual programs, studied outcomes for 3-5 year old students with autism spectrum disorder (ASD). Participants included 84 teachers and 302 students with ASD and their parents. CAP utilized specialized curricula and training components to implement…

Experimental evaluations that involve the educational system usually involve a hierarchical structure (students are nested within classrooms that are nested within schools, etc.). Concerns about contamination, where research subjects receive certain features of an intervention intended for subjects in a different experimental group, have often led…

A classroom randomized trial (n = 31 classrooms) was conducted using the Second Step Early Learning (SSEL) curriculum compared to usual curricula. Head Start and community preschool classrooms enrolling low income children were randomly assigned to deliver SSEL (n = 16) or usual curricula (n = 15). Data are reported for four year olds…

This randomized control study compares the efficacy of two response-to-intervention (RTI) models: (1) Dynamic RTI, which immediately refers grade 1 students with the weakest skills to the most intensive intervention supports (Tier 2 or Tier 3); and (2) Typical RTI, which starts all students in Tier 1 and after 8 weeks, decides whether students who…

The present study evaluates the effectiveness of Check & Connect (C&C) in a randomly assigned sample of students who were all receiving Communities in Schools (CIS) services. The research questions for the study include: Are there differences in attendance, academics, and behavior for CIS students who also receive C&C compared to…