Cluster Randomized Trial (CRT) designs are inherently multilevel and reflect the hierarchical structure of schools and the wider education system. To capture this multilevel nature, CRTs are commonly analysed using multilevel (or hierarchical) linear models. It is fairly common for CRT designs and analyses to include school and individual/pupil…

This paper presents evidence confirming the validity of the RD design in the Reading Recovery study by examining the ability of the RD design to replicate the 1st grade results observed in the original i3 RCT focused on short-term impacts. Over 1,800 schools participated in the RD study over all four cohort years. The RD design used cutoff-based…

Purpose: This research note reports on an unexpected negative finding related to behavior change in a controlled trial designed to test whether partner training improves the conversational skills of volunteers. Method: The clinical trial involving training in "Supported Conversation for Adults with Aphasia" utilized a single-blind,…

Some advances in bioethics regarding ethical considerations that arise in the context of medical research can also be relevant when thinking about the ethical considerations that arise in the context of SoTL research. In this article, I aim to bring awareness to two potential ethical challenges SoTL researchers might face when playing a dual role…

Evaluators of education interventions increasingly need to design studies to detect impacts much smaller than the 0.20 standard deviations that Cohen (1988) characterized as "small." For example, an evaluation of Response to Intervention from the Institute of Education Sciences (IES) detected impacts ranging from 0.13 to 0.17 standard…

Given the widespread use of non-experimental (NE) methods for assessing program impacts, there is a strong need to know whether NE approaches yield causally valid results in field settings. In within-study comparison (WSC) designs, the researcher compares treatment effects from an NE with those obtained from a randomized experiment that shares the…

The scientific rigor of education research has improved dramatically since the year 2000. Much of the credit for this improvement is deserved by Institute of Education Sciences (IES) policies that helped create a demand for rigorous research; increased human capital capacity to carry out such work; provided funding for the work itself; and…

The purpose of this paper is to present results of recent advances in power analyses to detect the moderator effects in Cluster Randomized Trials (CRTs). This paper focus on demonstration of the software PowerUp!-Moderator. This paper provides a resource for researchers seeking to design CRTs with adequate power to detect the moderator effects of…

More aggregate data on school performance is available than ever before, opening up new possibilities for applied researchers interested in assessing the effectiveness of school-level interventions quickly and at a relatively low cost by implementing comparative interrupted times series (CITS) designs. We examine the extent to which effect…

Conventionally, regression discontinuity analysis contrasts a univariate regression's limits as its independent variable, "R," approaches a cut point, "c," from either side. Alternative methods target the average treatment effect in a small region around "c," at the cost of an assumption that treatment assignment,…

Researchers often include covariates when they analyze the results of randomized controlled trials (RCTs), valuing the increased precision of the estimates over the potential of inducing small-sample bias when doing so. In this paper, we develop a sufficient condition which ensures that the inclusion of covariates does not induce small-sample bias…

Better integration of evidence-based practices (EBPs) in schools can improve access to effective youth mental health care. However, EBPs typically developed and tested within clinics may not be feasible for school-based implementation. We conducted a small randomized trial comparing Show Me FIRST, a brief intervention designed for efficient uptake…

Purpose: Key considerations in designing multilevel experimental studies are to efficiently use resources and to determine the sample size allocation such that designs have adequate statistical power. The utility of optimal sampling and power analysis results depends on accurate information about design parameters, such as intraclass correlations…

Background: It has become increasingly important to shift education research and programming from a deficit focused perspective to a positive youth development perspective in order to identify and promote factors that support students' positive functioning and outcomes. One outgrowth of positive youth development is Positive Mental Health (PMH),…

Background: Recent evidence suggests that pre-school children with co-occurring phonological speech sound disorder (SSD) and expressive language difficulties are at a higher risk of ongoing communication and literacy needs in comparison with children with these difficulties in isolation. However, to date there has been no systematic or scoping…