Evaluations should be issues driven, not methods driven. The starting point should be priority programs to be evaluated or policies to be tested. From this starting point, a list of evaluation questions is identified. For each evaluation question, the task is to identify the best available method for answering that question. Hence it is likely…

Critical race theory (CRT) is a relatively new theory that has been little used in evaluation practice. The authors explore an example of critical race evaluation working with James Comer's School Development Program. In this evaluation common tropes associated with CRT were used to construct a counternarrative of school reform. The authors…

With the vast array of contextual factors, pedagogical approaches, models of implementation, and purposes of education and training related to online learning, educators, learners, and the general public alike are seeking answers regarding utility and effectiveness of online learning. This article identifies and responds to many of the challenges…

The authors propose a comprehensive evaluation framework to prepare for evaluating school emergency management programs. This framework involves a logic model that incorporates Government Performance and Results Act (GPRA) measures as a foundation for comprehensive evaluation that complements performance monitoring used by the U.S. Department of…

Impact evaluations assess the degree to which changes in outcomes can be attributed to an intervention rather than to other factors. Such attribution requires knowing what outcomes would have looked like in the absence of the intervention. This counterfactual world can be inferred only indirectly through evaluation designs that control for…

Knowledge utilization is a field crossing many sectors, from agriculture, since the 1920s, to health care today. Evaluators have made long-standing contributions to understanding knowledge utilization. Different models or ways to think about knowledge utilization have evolved to reflect different perspectives, contexts, and stages of the process,…

A critical theory lens is used to explore the role of evaluation in youth development, a field aimed at recognizing youth as assets. A theory of change in the field is questioned for its emphasis on individual youth outcomes as programmatic outcome measures. A review of 209 evaluations of 131 programs in the Harvard Family Research Project's…

This chapter examines how factors both internal and external to Cooperative Extension have influenced its commitment and capability to assess the quality and impact of its programs. The authors begin by documenting how the nature of Extension programming has changed dramatically in response to societal needs over the course of the organization's…

The authors tell about their heterogeneous 91 person research and evaluation unit at an operating foundation in St. Paul, Minnesota. They focus on evaluation for program improvement, one of several purposes of studies they work on. The three authors write from their different manager positions within the unit. Included are the context of the…

The author considers whether current evaluation practice is of value to the Cooperative Extension System. Evaluation use and influence in Extension are characterized through literature review and examination of case examples informed by correspondence with study authors. The more than 675 evaluations examined consistently yielded specific and…

The authors review the unique characteristics of nonprofit organizations and explain how nonprofit organizations play a prominent role in providing today's public services. The authors describe how the field of professional evaluation developed in response to government performance-based management initiatives, noting that recent efforts at the…

A district evaluator in a large Texas district examines new challenges arising since implementation of No Child Left Behind, relating to (1) navigating competing requirements in state and federal accountability systems; (2) evaluating effectiveness of sanctions districts are required to address; (3) using scientifically based research (SBR) to…

The authors provide an overview of the Cooperative Extension System and its program evaluation challenges. Part of the historic land-grant system, Extension exists in all states and territories of the United States and is funded through federal, state, and local (usually county) appropriations, as well as competitive grants and other sources.…

The impact of No Child Left Behind (NCLB) is usually understood in relation to schools and districts, but the legislation has also affected community-based organizations that operate school-linked programs. This case study of an after-school program in California demonstrates how educational accountability systems that emphasize students' academic…

"Theory as Method: Small Theories of Treatments," by Mark W. Lipsey, is one of the most influential and highly cited articles to appear in "New Directions for Evaluation." It articulated an approach in which methods for studying causation depend, in large part, on what is known about the theory underlying the program. Lipsey discussed the benefits…