Realist evaluation (RE) is a research design increasingly used in program evaluation, that aims to explore and understand the influence of context and underlying mechanisms on intervention or program outcomes. Several methodological challenges, however, are associated with this approach. This article summarizes RE key principles and examines some…

Data Envelopment Analysis (DEA) is a widely adopted non-parametric technique for evaluating R&D performance. However, traditional DEA models often struggle to provide reliable solutions in the presence of data uncertainty. To address this limitation, this study develops a novel robust super-efficiency DEA approach to evaluate R&D…

This methods case study describes the design and implementation of a digital, multimodal journal from a yearlong program evaluation using self-study methods. The program evaluation focused on the author's transition of practice from elementary teacher to teacher educator and aimed to improve the author's practice. The design of the digital,…

While social programs are often assessed using short-term impact studies, longitudinal designs allow evaluators to capture change over time, identify longer-term outcomes, adapt instruments, and better understand participants in transition. A mixed methods design can be critical in understanding these dynamics; yet there is a lack of literature…

We revisit the merits of the retrospective pretest-posttest (RPP) design for repeated-measures research. The underutilized RPP method asks respondents to rate survey items twice during the same posttest measurement occasion from two specific frames of reference: "now" and "then." Individuals first report their current attitudes…

In this response, we first show that Simpson's proposed analysis answers a different and less interesting question than ours. We then justify the choice of prior for our Bayes factors calculations, but we also demonstrate that the substantive conclusions of our article are not substantially affected by varying this choice.

This report from the Career and Technical Education (CTE) Research Network Lead team provides final results from an evaluability assessment of CTE programs that feasibly could be evaluated using a rigorous experimental design. Evaluability assessments (also called feasibility studies) are used in education and other fields, such as international…

This paper describes the context, processes and issues experienced over 5 years in which a RCT was carried out to evaluate a programme for children aged 7-8 who were struggling with their reading. Its specific aim is to illuminate questions about the design of complex teaching approaches and their evaluation using an RCT. This covers the early…

A recent paper uses Bayes factors to argue a large minority of rigorous, large-scale education RCTs are "uninformative." The definition of "uninformative" depends on the authors' hypothesis choices for calculating Bayes factors. These arguably overadjust for effect size inflation and involve a fixed prior distribution,…

This response aims to critically evaluate the methodology and aims of the meta-analytic review written by Maggin and Johnson (2014). The present authors systematically provide responses for each of the original criticisms and highlight concerns regarding Maggin and Johnson's methodology, while objectively describing the current state of evidence…

In their National Bureau of Economic Research (NBER) working paper, "Does Gifted Education Work? For Which Students?" Card and Giuliano (C & G) made an enormous splash in not just gifted education but also the world (e.g., "The Washington Post," "The Atlantic," Five Thirty Eight). In this commentary, we highlight…

Treatment effect estimates from a "regression discontinuity design" (RDD) have high internal validity. However, the arguments that support the design apply to a subpopulation that is narrower and usually different from the population of substantive interest in evaluation research. The disconnect between RDD population and the…

Participatory Health Research is a collective term adopted globally for participatory action research in a health context. As an approach to research, it challenges current ways used within the health sciences to measure research impact as research, learning and action are integrated throughout the research process and dependent on context and…

Background: The Education Endowment Foundation (EEF) is an independent charity that was established in 2011 with the explicit aim of breaking the link between family income and educational achievement in England. Over the seven years since its inception, EEF has contributed to the existing evidence base by funding over one hundred randomised…

Card and Giuliano conducted a regression discontinuity study in a large Florida school district to investigate the magnitude of academic benefits of the district's gifted program. They found that for children identified as gifted through an intelligence test, the program provided few or no benefits. But children who were admitted to the gifted…