When designing cluster-randomized trials (CRTs), one important consideration is determining the proper sample sizes across levels and treatment conditions to cost-efficiently achieve adequate statistical power. This consideration is usually addressed in an optimal design framework by leveraging the cost structures of sampling and optimizing the…

Background: Multilevel randomized controlled trials (MRCTs) have been widely used to evaluate the causal effects of educational interventions. Traditionally, educational researchers and policymakers focused on the average treatment effects (ATE) of the intervention. Recently there has been an increasing interest in evaluating the heterogeneity of…

Combining observational and experimental data for causal inference can improve treatment effect estimation. However, many observational data sets cannot be released due to data privacy considerations, so one researcher may not have access to both experimental and observational data. Nonetheless, a small amount of risk of disclosing sensitive…

Longitudinal randomized controlled trials generally assign individuals randomly to interventions at baseline and then evaluate how differential average treatment effects evolve over time. This study shows that longitudinal settings could benefit from "Recurrent Individual Treatment Assignment" ("RITA") instead, particularly in…

We perform a large-scale randomized controlled trial to evaluate the potential of machine learning-based instruction sequencing to improve memorization while allowing the learners the freedom to choose their review times. After controlling for the length and frequency of study, we find that learners for whom a machine learning algorithm determines…

Background: An efficacy study based on a randomized controlled trial (RCT) was designed to evaluate the impact of ASSISTments on math learning among middle school students, including those from high-needs rural schools. Despite extensive recruitment efforts in 2021 (during the pandemic), only a few schools and their teachers agreed to participate,…

This is a technical report detailing the analytic methods used to assess the effects of a virtual professional learning community (vPLC)-enhanced ASSISTments intervention. The report uses a quasi-experimental design and partially nested regression model to account for clustering and reveals that while overall effects were positive but not…

Randomized controlled trials (RCTs) admit unconfounded design-based inference--randomization largely justifies the assumptions underlying statistical effect estimates--but often have limited sample sizes. However, researchers may have access to big observational data on covariates and outcomes from RCT non-participants. For example, data from A/B…