Discusses ethical challenges faced by evaluators, especially those who have gotten into problems while conducting the evaluation. Introduces the other articles in this theme section. (SLD)

Notes that, very early in program planning, budgets are estimated for major program components, including evaluation. Proposes heuristic for establishing reasonable evaluation estimates in initial planning. Describes logical sources of information from which to construct such a heuristic. Recommends that, for typical programs and projects, initial…

Reflects on an evaluation that aspired to be inclusive but generally failed to provide a backdrop for a discussion of inclusive evaluation. Identifies issues of absence of significant stakeholders, making of values by method, and the limited authority of the evaluation. Shows how easily deliberative intentions are distorted. (SLD)

Challenges the assumption that evaluation should aim for fostering deliberative democracy. A modest effort by an evaluator to contribute to deliberative democracy is defensible, but vigorous political advocacy violates social expectation. (Author/SLD)

The gold standard in making causal inference on program effects is a randomized trial. Most randomization designs in education randomize classrooms or schools rather than individual students. Such "clustered randomization" designs have one principal drawback: They tend to have limited statistical power or precision. This study aims to…

Career development practitioners understand that the value of career and employment services extends beyond attachment to the labor market. However, "soft" outcomes such as increased self-efficacy and improved ability to manage transitions are difficult to measure. Convincing funders and policy makers of the significance of such outcomes…

This paper describes three multigrade school reforms in Latin America: (1) Colombia's "Escuela Nueva", (2) Guatemala's "Nueva Escuela Unitaria", and (3) Chile's MECE-Rural. Each reform endowed primary teachers and students with special training and instructional materials, and encouraged new kinds of instruction in rural…

Values and beliefs underlying assumptions concerning the independent evaluations of persons and programs involved in the educational process are discussed. Particular emphasis is placed on M. Scriven's treatise on bias in evaluation (1976). Works by E. R. House, R. E. Stake, L. Stenhouse, M. Parlett and D. Hamilton, and D. L. Stufflebeam are also…

Outlines a scenario in which an evaluator wants to ask questions of teenagers that may pose ethical problems, or may be perceived as posing such problems. The hypothetical situation is the evaluation of a life skills program aimed at reducing the teen birth rate. (SLD)

Taking the position that science is not value-free has certain consequences. There are both risks and rewards inherent when evaluators relinquish an objective, non-involved role for a more subjective, partisan role in an evaluation where issues of class, gender, and race are intertwined. It is important for evaluators to consider how their…

Suggests a strategy for dealing with the situation in which evaluators wish to ask teenage participants in a life skills program questions that might be seen as controversial. The strategy relies on an inclusive planning committee, a low-key poll to provide background information, and a foot in the door opening to start the process. (SLD)

Suggests that the key issue in the evaluation problem scenario is not so much an ethical problem as it is a strategic problem. The evaluator needs to be educator, change agent, and organizational development practitioner to make this evaluation a success. (SLD)

Describes and critiques medical school rankings by "U.S. News & World Report" on methodologic and conceptual grounds, arguing that the annual evaluation falls short in both areas. Presents three alternative categories of program quality indicators. Concludes that the rankings have no practical value and fail to meet standards of…

A report from the School Choice Demonstration Project examines issues concerning the funding formula used for the Milwaukee Parental Choice Program (MPCP). It finds that the program generates a net saving to taxpayers in Wisconsin but imposes a significant fiscal burden on taxpayers in Milwaukee. However, these findings depend significantly on how…

Treatment integrity (also referred to as "treatment fidelity," "intervention integrity," and "procedural reliability") is an important methodological concerning both research and practice because treatment integrity data are essential to making valid conclusions regarding treatment outcomes. Despite its relationship to validity, treatment…