Background: Mission US is a series of interactive first-person role-playing history games and curricular materials that address a critical problem: students lack fundamental knowledge of our nation's history. According to the most recent National Assessment of Educational Progress (NAEP, 2018), only 13% of Grade 8 students were proficient on the…

Recent methods have sought to improve precision in randomized controlled trials (RCTs) by utilizing data from large observational datasets for covariate adjustment. For example, consider an RCT aimed at evaluating a new algebra curriculum, in which a few dozen schools are randomly assigned to treatment (new curriculum) or control (standard…

Narration has been shown to be a foundational skill for literacy development in school-age children. Elementary teachers routinely conduct classroom lessons that focus on reading decoding and comprehension, but they rarely provide instruction in oral narration (Hall et al., 2021). This multisite randomized controlled trial was designed to…

Randomized A/B tests within online learning platforms represent an exciting direction in learning sciences. With minimal assumptions, they allow causal effect estimation without confounding bias and exact statistical inference even in small samples. However, often experimental samples and/or treatment effects are small, A/B tests are underpowered,…

Background: Randomised trials have been on the rise in social policy over the last decade and a half, particularly in areas working with young people and vulnerable adults. Informed consent is an important principle for ethics committees governing research conducted by universities. Aims and objectives: We consider the arguments for and against…

Over the last twenty years, education researchers have increasingly conducted randomised experiments with the goal of informing the decisions of educators and policymakers. Such experiments have generally employed broad, consequential, standardised outcome measures in the hope that this would allow decisionmakers to compare effectiveness of…

Purpose: In a companion paper, we found no statistical reason to favor percentage syllables stuttered (%SS) over parent-reported stuttering severity as a primary outcome measure for clinical trials of early stuttering. Hence, considering the logistical advantages of the latter measure, we recommended parent-reported stuttering severity for use as…

This article discusses estimation of average treatment effects for randomized controlled trials (RCTs) using grouped administrative data to help improve data access. The focus is on design-based estimators, derived using the building blocks of experiments, that are conducive to grouped data for a wide range of RCT designs, including clustered and…

The What Works Clearinghouse (WWC) identifies existing research on educational interventions, assesses the quality of the research, and summarizes and disseminates the evidence from studies that meet WWC standards. The WWC aims to provide enough information so educators can use the research to make informed decisions in their settings. This…

Background: When RCTs are not feasible and time series data are available, panel data methods can be used to estimate treatment effects on outcomes, by exploiting variation in policies and conditions over time and across locations. A complication with these methods, however, is that treatment timing often varies across the sample, for example, due…

The National Board of Certification and Recertification for Nurse Anesthetists (NBCRNA) conducted a one-year research study comparing performance on the traditional continued professional certification assessment, administered at a test center or online with remote proctoring, to a longitudinal assessment that required answering quarterly…

Randomized A/B tests within online learning platforms enable us to draw unbiased causal estimators. However, precise estimates of treatment effects can be challenging due to minimal participation, resulting in underpowered A/B tests. Recent advancements indicate that leveraging auxiliary information from detailed logs and employing design-based…

The current situation due to the COVID-19 pandemic has had huge consequences in all aspects of our lives, including the development of research activities. Studies involving face-to-face interaction with people, such as randomized controlled trials, were the first affected. The objective of this article is to present the insights on challenges…

At every stage since the start of the pandemic, decisions across education and childcare have been informed by the scientific and medical evidence -- both on the risks of coronavirus (COVID-19) infection, transmission and illness, and on the known risks to children and young people not attending education settings -- balancing public health and…

Randomized control trials are considered the pinnacle for causal inference. In many cases, however, randomization of participants in social work research studies is not feasible or ethical. This paper introduces the co-twin control design study as an alternative quasi-experimental design to provide evidence of causal mechanisms when randomization…