Purpose: This research note reports on an unexpected negative finding related to behavior change in a controlled trial designed to test whether partner training improves the conversational skills of volunteers. Method: The clinical trial involving training in "Supported Conversation for Adults with Aphasia" utilized a single-blind,…

Some advances in bioethics regarding ethical considerations that arise in the context of medical research can also be relevant when thinking about the ethical considerations that arise in the context of SoTL research. In this article, I aim to bring awareness to two potential ethical challenges SoTL researchers might face when playing a dual role…

Evaluators of education interventions increasingly need to design studies to detect impacts much smaller than the 0.20 standard deviations that Cohen (1988) characterized as "small." For example, an evaluation of Response to Intervention from the Institute of Education Sciences (IES) detected impacts ranging from 0.13 to 0.17 standard…

Given the widespread use of non-experimental (NE) methods for assessing program impacts, there is a strong need to know whether NE approaches yield causally valid results in field settings. In within-study comparison (WSC) designs, the researcher compares treatment effects from an NE with those obtained from a randomized experiment that shares the…

The scientific rigor of education research has improved dramatically since the year 2000. Much of the credit for this improvement is deserved by Institute of Education Sciences (IES) policies that helped create a demand for rigorous research; increased human capital capacity to carry out such work; provided funding for the work itself; and…

Teacher professional development (TPD) programs are increasingly using video recordings of teaching practice to develop teacher capacity and foster student learning. However, consensus has yet to be reached about how to utilize video recordings in TPD for physical education (PE) teachers. We used semi-structured interviews and evaluations of PE…

Background/Context: When a researcher tests an educational program, product, or policy in a randomized controlled trial (RCT) and detects a significant effect on an outcome, the intervention is usually classified as something that "works." When the expected effects are not found, however, there is seldom an orderly and transparent…

This paper presents evidence confirming the validity of the RD design in the Reading Recovery study by examining the ability of the RD design to replicate the 1st grade results observed in the original i3 RCT focused on short-term impacts. Over 1,800 schools participated in the RD study over all four cohort years. The RD design used cutoff-based…

More aggregate data on school performance is available than ever before, opening up new possibilities for applied researchers interested in assessing the effectiveness of school-level interventions quickly and at a relatively low cost by implementing comparative interrupted times series (CITS) designs. We examine the extent to which effect…

Conventionally, regression discontinuity analysis contrasts a univariate regression's limits as its independent variable, "R," approaches a cut point, "c," from either side. Alternative methods target the average treatment effect in a small region around "c," at the cost of an assumption that treatment assignment,…

Researchers often include covariates when they analyze the results of randomized controlled trials (RCTs), valuing the increased precision of the estimates over the potential of inducing small-sample bias when doing so. In this paper, we develop a sufficient condition which ensures that the inclusion of covariates does not induce small-sample bias…

A recent paper uses Bayes factors to argue a large minority of rigorous, large-scale education RCTs are "uninformative." The definition of "uninformative" depends on the authors' hypothesis choices for calculating Bayes factors. These arguably overadjust for effect size inflation and involve a fixed prior distribution,…

Youth with ASD often show limited or atypical empathic responsiveness. The direct effects of social skills interventions on enhancing empathic responsiveness is unknown. Data from a randomized controlled trial were used to investigate whether a Theory of Mind training improves the empathic responsiveness, measured through structured observations.…

Two non-U.S. quasi-experimental studies reported Active Support training was associated with increased engagement in individuals with IDD, but no randomized controlled trials (RCTs) exist. We evaluated effects of Active Support training on staff assistance, and social and nonsocial engagement in 75 individuals with intellectual and developmental…

The significance of a process evaluation for understanding randomised controlled trials (RCTs) of complex teaching interventions in a classroom setting is discussed in relation to the evaluation of the Integrated Group Reading (IGR) programme trial. This process evaluation is based on realist principles and draws on different data sources. The…