Why conclusions of many outcome evaluations do not stand up to scrutiny is discussed, drawing on examples from evaluations of drug abuse prevention programs. Factors that undermine these studies are largely the result of social-structural problems that influence the design and implementation of the research. (SLD)

This article describes an ongoing survey that illustrates how evaluators, working with program managers, have effectively used client-based data to monitor performance in the Social Security Administration. The value of client-based data outweighs the limitations and problems collecting it. Customer perceptions are a critical barometer of quality.…

Evaluation findings of the National Institute on Alcohol Abuse and Alcoholism (NIAAA) Community Demonstration Program show few clear indications of the effectiveness of case management approaches. This article identifies phenomena that explain the lack of positive effects and suggests that evaluation problems may explain apparent negative…

Statistical identification of employees with unusual sick leave patterns can result in false-positive identifications. A computer simulation model was used to evaluate alternate methods of unusual sick leave detection for the New York State Department of Health. Results were used to refine identification parameters and modify supervisory…

Evaluation is important to public education and training programs if the United States is to be competitive in a global economy. Six major factors have had an impact on evaluation efforts: decreased financial resources, increased public dissatisfaction, changed management approaches, enhanced evaluation methods and procedures, redesigned…

Program design can aid problem-solving or decision-making tasks faced by evaluators. It can be used to facilitate evaluation planning through the definition of program boundaries and the generation and focusing of evaluation questions. Further, it can provide evaluation information about theoretical, historical, and political contexts of a…

This briefing report on program evaluation notes that the new wave of family-centered, integrated services is forcing the field to rethink its methodology, and that the move towards outcome-driven reforms requires carefully defined outcomes and ways to measure them. Highlights of a seminar meeting held to explore these issues are included. The…

This report documents the National Assessment of Educational Progress (NAEP) special pilot study of group assessment. In 1994, NAEP administered U.S. History projects to a limited number of students. The purpose of this study was to investigate the feasibility of group assessment, and to gain practical experience in the design, development,…

This report focuses on the question of how to evaluate workplace education programs. Chapter 1 outlines the various purposes that those involved in workplace education programs have defined for workplace education and describes program models in place to meet those goals. Chapter 2 sets out three scenarios for evaluation at present: informal,…

A researcher's experiences in providing technical support to 10 schools involved in evaluation of their school-based management programs is provided in this report. Methodology involved principal interviews, attitudinal staff surveys, and school self-evaluations of the 10 participating schools. Recommendations for improving the self-evaluation…

Problems in conducting an outcome evaluation under uncertain knowledge of effects are addressed. This evaluation was designed to obtain outcome assessments of a National Science Foundation Workshop, which was working with 20 high school chemistry teachers and 20 high school biology teachers to stimulate effective teaching, assist teachers in…

The choice of methods is part of the overall evaluation design process. The process consists of the following steps: (1) analyzing the problem context; (2) asking a few general questions; (3) selecting the methods (strategies) to use; and (4) selecting the specific techniques (tactics) to use. To operate successfully the evaluator needs to know…

One hundred of the largest school districts from seven regions were surveyed to scrutinize local teacher evaluation policies and practices, to determine the efficiency of local evaluation instruments and procedures, and to compare local teacher evaluation with state teacher evaluation programs. The Teacher Evaluation Practices Survey (TEPS) was…

Examples of the difficulties that public school program evaluators can have in explaning perplexing results are provided. Arguments abound over the best way to report a finding that has multiple interpretations or presentation options, and the enigma of realizing that one way to report a finding is not the only way, or even necessarily the best…

Twenty-five years of evaluation research are reviewed. Findings show that evaluations of social programs have been negative. Policymakers often blamed the evaluators when it was not clear why certain programs had failed. There is hope for a better working relationship as evaluators adopt better methods and social policy becomes more rational. (VM)