There has been increased focus within the evaluation field on the value of community-engaged research (CER) methods; however, CER is not easily seen as compatible with experimental evaluation methods (or randomized controlled trial, RCT). For instance, in an experimental design, researchers leverage randomization to create a counterfactual; this…

In recent years, countries including the UK and USA have seen advancements in the use of Randomised Controlled Trials in education, progress that has not been mirrored in Ireland. Ireland does not have a strong tradition of using experimental or quasi-experimental evaluation designs for monitoring and evaluation of education policy despite…

The Campbellian validity typology has been used as a foundation for outcome evaluation and for developing evidence-based interventions for decades. As such, randomized control trials were preferred for outcome evaluation. However, some evaluators disagree with the validity typology's argument that randomized controlled trials as the best design…

Although experimental evaluations have been labeled the "gold standard" of evidence for policy (U.S. Department of Education, 2003), evaluations without an analysis of costs are not sufficient for policymaking (Monk, 1995; Ross et al., 2007). Funding organizations now require cost-effectiveness data in most evaluations of effects. Yet,…

Parenting programs have been shown to be effective in preventing and reducing externalising problems in young children. Despite their efficacy, the low rate of initial parental engagement in these programs is a major challenge for clinicians and researchers. Few studies have examined factors associated with rates of initial engagement in parenting…

Meta-analyses are systematic summaries of research that use quantitative methods to find the mean effect size (standardized mean difference) for interventions. Critics of meta-analysis point out that such analyses can conflate the results of low- and high-quality studies, make improper comparisons and result in statistical noise. All these…

Many autistic children access some form of early intervention, but little is known about the value for money of different programs. We completed a scoping review of full economic evaluations of early interventions for autistic children and/or their families. We identified nine studies and reviewed their methods and quality. Most studies involved…

This paper seeks to raise awareness among educational researchers and practitioners of some significant weaknesses and internal contradictions of randomised control trials (RCTs). Although critiques throughout the years from education scholars have pointed to the detrimental effects of this experimental approach on education practice and values,…

In multisite trials, learning about treatment effect variation across sites is critical for understanding where and for whom a program works. Unadjusted comparisons, however, capture "compositional" differences in the distributions of unit-level features as well as "contextual" differences in site-level features, including…

Background: Randomised controlled trials (RCTs) have long been considered the gold standard in education research. Federal funds are allocated to evaluations that meet What Works Clearinghouse standards; RCT designs are required in order to meet these standards without reservations. Schools seek out interventions that are research based, in other…

This article offers a case example of how experimental evaluation methods can be coupled with principles of design-based implementation research (DBIR), improvement science (IS), and rapid-cycle evaluation (RCE) methods to provide relatively quick, low-cost, credible assessments of strategies designed to improve programs, policies, or practices.…

In multisite trials, learning about treatment effect variation across sites is critical for understanding where and for whom a program works. Unadjusted comparisons, however, capture "compositional" differences in the distributions of unit-level features as well as "contextual" differences in site-level features, including…

In 2015, Abt Associates received a grant from the Institutes for Education Sciences (IES) for a five-year "Development and Innovation" study of PTC. The purposes of the study were to gauge progress in implementing PTC and to develop and test improvements where needed. Fein et al. (2020) summarize the IES study's approach and findings. A…

Education decisionmakers need access to the best evidence about the effectiveness of education interventions, including practices, products, programs, and policies. It can be difficult, time consuming, and costly to access and draw conclusions from relevant studies about the effectiveness of interventions. The What Works Clearinghouse (WWC)…

Background: While implementation studies in aphasia management have shown promising improvements to clinical practice, it is currently unknown if aphasia implementation outcomes are sustained and what factors may influence clinical sustainability. Aims: To evaluate the sustainment (i.e., sustained improvement of aphasia management practices and…