Realist evaluation (RE) is a research design increasingly used in program evaluation, that aims to explore and understand the influence of context and underlying mechanisms on intervention or program outcomes. Several methodological challenges, however, are associated with this approach. This article summarizes RE key principles and examines some…

In this response, we first show that Simpson's proposed analysis answers a different and less interesting question than ours. We then justify the choice of prior for our Bayes factors calculations, but we also demonstrate that the substantive conclusions of our article are not substantially affected by varying this choice.

This paper describes the context, processes and issues experienced over 5 years in which a RCT was carried out to evaluate a programme for children aged 7-8 who were struggling with their reading. Its specific aim is to illuminate questions about the design of complex teaching approaches and their evaluation using an RCT. This covers the early…

Treatment effect estimates from a "regression discontinuity design" (RDD) have high internal validity. However, the arguments that support the design apply to a subpopulation that is narrower and usually different from the population of substantive interest in evaluation research. The disconnect between RDD population and the…

Background: The Education Endowment Foundation (EEF) is an independent charity that was established in 2011 with the explicit aim of breaking the link between family income and educational achievement in England. Over the seven years since its inception, EEF has contributed to the existing evidence base by funding over one hundred randomised…

The effective evaluation of correctional programs is critically important. However, research in corrections rarely allows for the randomization of offenders to conditions of the study. This limitation compromises internal validity, and thus, causal conclusions can rarely be drawn. Increasingly, researchers are employing propensity score matching…

Over the past few years, the focus group method has assumed a very important role as a method for collecting qualitative data in social and behavioural science research. This article elucidates theoretical and practical problems and prospects associated with the use of focus groups as a qualitative research method in social and behavioural science…

The focus of this chapter is on the evidence to date that documents the impact of PDS engagement on four specific groups of participants: veteran P-12 teachers, university faculty, P-12 school principals, and parents and community members. In reviewing the literature concerning the impact of PDS engagement on these four groups, the authors'…

This paper describes the analysis of the National Science Foundation (NSF) Math and Science Partnership (MSP) program's January 2008 Learning Network Conference (LNC). This study focuses on the methodologies used by the MSP community to generate evidence and seeks to understand topics of interest to the 320 LNC participants. The data set for this…

This article provides a brief summary of the seven articles in this special issue through the lens of the concept of "caring reflexivity". In joining "caring" and "reflexivity", we deepen the conversation about what constitutes reflexivity, encouraging an explicit focus on the relational. Revisiting the first article,…

This article discusses the difficulties in conducting intervention research or evaluating intervention programs in a school setting. In particular, the problems associated with randomization and obtaining control groups are examined. The use of quasi-experimental designs, specifically a paired comparison design using the individual as his or her…

This article discusses the use of propensity scoring in experimental program evaluations to estimate impacts for subgroups defined by program features and participants' program experiences. The authors discuss estimation issues and provide specification tests. They also discuss the use of an overlooked data collection design--obtaining predictions…

In this same "Kappan" Issue, Walberg and Greenberg mistakenly allege that Success for All was developed for ulterior (financial) motives, is based on questionable research, and has had poor results. Literacy achievement has not risen for 70 years. Success for All is no panacea but is tackling this problem. (10 references) (MLH)

This article describes the potential measurement problem in analyzing costs and benefits of vocational education so that policymakers and educators can distinguish between proper use and misuse of this evaluation technique. (Author/CT)

This paper provides a critical analysis of New Zealand's Performance Based Research Fund (PBRF). The first section sketches the development and implementation of the PBRF. The second section evaluates the scheme, concentrating on three themes: the relationship between privatization, competition and research performance; the standardization of…