I explore how "low-quality evidence" from program performance might still be useful in decision-making. Conceptually, a local government named "Here" is motivated to consider a program from "Elsewhere" that seems to show year-over-year exemplary performance. Here must manage five sources of uncertainty about whether…

The recognition that researcher discretion coupled with unconscious biases and motivated reasoning sometimes leads to false findings ("p-hacking") led to the broad embrace of study preregistration and other open-science practices in experimental research. Paradoxically, the preregistration of quasi-experimental studies remains uncommon…

The study dealt with a research environment in which the treatment effect is heterogeneous, and in which individuals use their assessments of the treatment effect to decide whether or not to enroll in an intervention program. In this article, a new methodology is proposed for examining the validity of the specified research environment in a given…

This article discusses the use of propensity scoring in experimental program evaluations to estimate impacts for subgroups defined by program features and participants' program experiences. The authors discuss estimation issues and provide specification tests. They also discuss the use of an overlooked data collection design--obtaining predictions…

The Intensive Protective Supervision Project (IPSP) is a juvenile justice program in North Carolina designed to assist status offenders. Several legislative and programmatic influences antedated the current program. This article provides a brief summary of three major events affecting the evaluation of the IPSP. (TJH)

In 2003, the Office of Juvenile Justice and Delinquency Prevention (OJJDP) provided Florida's Department of Juvenile Justice (DJJ) with a $3.5 million grant to design and implement a faith- and community-based program for incarcerated juveniles. Florida's experience developing a completely new program has provided valuable lessons regarding…

Presents a case study of the development and implementation of the "results framework," an innovative planning and evaluation tool that is becoming a standard for U.S. Agency for International Development projects. Explains the advantages of this evaluation and monitoring approach. (SLD)

Until the past few years, our nation's approach to designing federal programs for preschool-age children lacked coherence and paid little attention to what had worked (and not worked) in the past. In this article, the authors propose that credible information useful for designing effective programs will require the ongoing, systematic development…

The methodology and design of the process evaluation system of the Pawtucket Heart Health Program (an 11-year project for changing/documenting change in cardiovascular disease risk status of a northeastern city's residents) are described. Examples of the system's use are provided. The maintenance of such a system without substantial evaluation…

In contrast to the prevailing image of monitoring systems as technical systems, it is proposed that they should rather be conceived of as social endeavors at exchanging information. Drawing on the monitoring and evaluation framework of Cornford, Doukidis, and Forster, the concept of information agreement is suggested as a way of assessing the…

An evaluation system was designed which directly addresses federal legislation for accountability in vocational education programs. Six criteria were included; three focus on interim evidence of project success, and three address final evidence of project effectiveness. (Author/GK)

The 11 programs and sites participating in the Bureau of Justice Assistance (BJA) Intensive Supervision Demonstration Project are described. The demonstration was designed to assess the effects and costs of sentencing convicted felons to community-based programs. The evaluation by the Rand Corporation and data collection methods and random…

This article outlines an approach for developing and applying computerized decision support systems to the formulation and evaluation of public policy. To meet the challenge of financial resource limitations, new management systems must be developed to improve both governmental efficiency and decision-making effectiveness. (Author/BS)

Describes the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) and reviews and assesses research on its impact. Recommends policy reforms and randomized experiments to evaluate the program and any changes made to it. Weaknesses in research to date make the real impacts of WIC unclear. (Contains 79 references.) (SLD)

Explores sequential assignment (SA) and treatment-as-usual (TAU) as two approaches to overcome many of the obstacles of random assignment in field studies of treatment efficacy. Substituting SA and TAU for the standard random assignment may introduce methodological impurities including certain limited biases. (SLD)