Background: Recent reviews suggest that many plausible programs are found to have at best small impacts not commensurate with their cost, and often have no detectable positive impacts at all. Even programs with initial rigorous impact evaluation (RIE) that show them to be effective often fail a second test with an expanded population or at…

Recent meta-analyses of drug-court studies recognized the poor methodological quality of the evaluations, with only a few being randomized. This article critiques the design of the randomized studies from a statistical perspective. Learning points are identified for future drug-court studies and are applicable to evaluations both of other…

Although experiments are viewed as the gold standard for evaluation, some of their benefits may be lost when, as is common, outcomes are not defined for some sample members. In evaluations of marriage interventions, for example, a key outcome--relationship quality--is undefined when a couple splits up. This article shows how treatment-control…

This article explores the statistical methodologies used in demonstration and effectiveness studies when the treatments are applied across multiple settings. The importance of evaluating and how to evaluate these types of studies are discussed. As an alternative to standard methodology, the authors of this article offer an empirical binomial…

Some questions about the relationship between program evaluation and forecasting are raised. Contrasting the two fields in terms of mind-set, purpose, problems, advantages, and use shows that although their modes of inquiry are indeed very different in many ways, there is interdependence between the two in several areas. (Author/JAZ)

Probabilistic risk assessment has been promoted within the Nuclear Regulatory Commission as a means of judging risk to the public and of determining regulatory measures. Interviews with engineers and other technically trained personnel reveal the difficulties created by expectations that this form of assessment should be applied. (TJH)

One source of bias, regression artifacts, in program evaluations that use regression discontinuity design is identified, and its importance is discussed. A method is suggested for its correction. Although use of this method does not solve the problem of selection bias, it does improve the reliability of nonexperimental methods. (SLD)

Describes the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) and reviews and assesses research on its impact. Recommends policy reforms and randomized experiments to evaluate the program and any changes made to it. Weaknesses in research to date make the real impacts of WIC unclear. (Contains 79 references.) (SLD)

A broad set of ethical principles is examined to determine their relevance for evaluation. The five principles of K. S. Kitchener (autonomy, nonmaleficience, beneficence, justice, and fidelity) are relevant to evaluation and are fully represented in the evaluation standards of the Joint Committee on Standards for Educational Evaluation. (SLD)

The role of evaluation in electric utility demand-side management (DSM) programs is explored, including the importance of evaluation in determining financial incentives that a company will earn for DSM programs. A hypothetical example is presented of the types of problems that might arise when evaluations are subject to litigation. (SLD)

Six potential problems with the use of randomized experiments to evaluate programs in the field are addressed. Problems include treatment dilution, treatment contamination or confounding, inaccurate case flow and power estimates, violations of the random assignment processes, changes in the environmental context, and changes in the treatment…

Dilemmas inherent in the attempt to measure and evaluate informal supports available to individuals in need of social care are illustrated through a study of 400 elderly persons in Jerusalem. Practical guidelines for evaluation are presented. (SLD)

The validity of true experiments is threatened by a class of self-report biases that affect all respondents at pretest, but which are diminished by treatment. Four of these inaccurate self-evaluation biases are discussed. Means of detection include external criteria, special conditions of measurement, and retrospective pretests. (TJH)

The use of qualitative evaluation methods is explored as a means of overcoming constraints and complexities of evaluation of development programs. A case study demonstrating the application of qualitative analysis to monitoring and evaluation of the Basic Minimum Needs program in Thailand is presented. (SLD)

W. M. K. Trochim and others defend the record of the regression-discontinuity (RD) design and blur the statistical tests for treatment effect. Their Monte Carlo results show the problematic nature of RD and its potential bias. New testing strategies and restrictions for the application of RD are proposed. (SLD)