Realist evaluation (RE) is a research design increasingly used in program evaluation, that aims to explore and understand the influence of context and underlying mechanisms on intervention or program outcomes. Several methodological challenges, however, are associated with this approach. This article summarizes RE key principles and examines some…

Data Envelopment Analysis (DEA) is a widely adopted non-parametric technique for evaluating R&D performance. However, traditional DEA models often struggle to provide reliable solutions in the presence of data uncertainty. To address this limitation, this study develops a novel robust super-efficiency DEA approach to evaluate R&D…

Shortly after we published "Addressing Flawed Research in Developmental Education" (2012) in the "Journal of Developmental Education," Thomas Bailey, Shanna Smith Jaggars, and Judith Scott- Clayton from the Community College Research Center (CCRC) wrote a response rebutting several of our claims. Though their response corrects…

As a result of the 1998 reauthorization of Head Start, the Department of Health and Human Services conducted a national evaluation of the Head Start program. The goal of Head Start is to improve the school readiness skills of low-income children in the United States. There is a substantial body of experimental and correlational research that has…

This "What Works Clearinghouse Procedures and Standards Handbook (Version 3.0)" provides a detailed description of the standards and procedures of the What Works Clearinghouse (WWC). The remaining chapters of this Handbook are organized to take the reader through the basic steps that the WWC uses to develop a review protocol, identify…

Over the past few years, the focus group method has assumed a very important role as a method for collecting qualitative data in social and behavioural science research. This article elucidates theoretical and practical problems and prospects associated with the use of focus groups as a qualitative research method in social and behavioural science…

A re-analysis of evaluation data concerning the Emergency School Aid Act shows that findings of the original evaluators are open to multiple interpretations. (JKS)

Professional evaluators are often called upon to analyze data produced by a catastrophically inadequate evaluation design. A remedial strategy involving diagnosis of error, application of a corrective procedure and sensitization of program personnel of the need for a more sophisticated stance, is proposed and illustrated with a case study.…

Discusses problems encountered in evaluating a mass drug education program including instrument selection, logistical constraints, and data analysis. Offers suggestions as to program evaluation improvement. (Author/RC)

This paper is devoted to an analysis of the "Standards for Evaluation of Educational Programs, Projects, and Materials" as they apply to qualitative data. The most difficult ethical problem arises from the conflict between the right to privacy of research subjects and the right to know of audiences. There are two technical points in the…

The financial aspects of educational programs are discussed in this commentary on the evaluation of the Emergency School Aid Act. Budget and expenditure data must be examined closely when evaluating educational programs. (JKS)

The estimation of the "size of effect" of educational programs is a difficult problem in program evaluation. It is argued that the intentions of the program and the nature of the measures must be known in order to estimate program effects. (JKS)

This response to TM 505 708 briefly reviews the five major points of that article, and adds seven points that evaluators should consider when preparing reports. Illustrations are taken from Project Follow Through. (BW)

Hypothesis generation and testing is outlined as an additional domain for program evaluators. Program evaluation involves a thorough analysis of the processes that contribute to change (or a lack of change) among program recipients. This process of change is analyzed in two ways: (1) treating programs as naturally occurring field studies; and (2)…

Data from 86 Early Childhood Education (ECE) projects in Oregon were analyzed using methods prescribed by the Value Added model for ECE evaluation. The extent to which appropriate model implementation procedures were followed is unknown. Thus, the results represent an empirical account of the model's estimates of project impact, rather than a…