Provides strategies for constructing theories of theory-based evaluation and provides examples in the field of public health. Techniques are designed to systematize and bring objectivity to the process of theory construction. Also introduces a framework of program theory. (SLD)

Considers what a responsible test developer would do to gain information to support the consequential basis of validity for a test early in the development. How the consequential basis of validity of the program would be monitored and reported during the life of the program is examined. The validity of the ACT Assessment is considered as if it…

Describes efforts to build a worldwide evaluation community, noting events that resulted in an almost 200% growth in the number of regional and national evaluation organizations in slightly more than 1 year. Profiles some national and regional evaluation organizations that participated in a recent evaluation meeting and suggests some alternative…

Provides a basic formula for calculating the return-on-investment of training: (1) collect evaluation data; (2) isolate effects of training from other factors; (3) convert results to monetary benefits; (4) total costs of training; and (5) compare the benefits with the cost. (JOW)

Explores the relative strengths and weaknesses of program theory as a tool for inferring causality and outlines a five-stage approach that makes increased use of inductively built program theories as it takes more deliberate account of the varying levels of certainty that are required for evaluative conclusions. (Author/SLD)

Even simple program theory evaluations could be used in meta-analysis to provide benefits. Explores several ways to combine program theory evaluation and meta-analysis, using a hypothetical model and an actual example from a federal funding program. Outlines four potential benefits to incorporating program theory evaluation into meta-analysis.…

Describe the role of an expert panel in setting priorities for evaluative work. An expert panel with high credibility inside and outside the organization that is at arms length to both the organization and external stakeholders can bypass many communications filters and provide a distilled and clear reading of stakeholder concerns. (SLD)

Presents a conceptual framework for putting program theory into operation. The framework incorporates variables that reflect theoretical concepts and implementation issues addressed in program evaluation. It organizes variables into input, process, and output categories. Discusses implications for program evaluation. (SLD)

Provides managers with specific guidance on developing and using logic models. For the last 20 years logic models have been used largely in program evaluations to chart what should have happened and what did or did not occur as intended. Logic models start with inputs and work through processes to the desired end state, whether output or outcome.…

Makes the case that both internal and external evaluation are needed in the assessment of educational systems. Suggests some conditions have to be met to establish a constructive dialogue between internal and external evaluation. (SLD)

Discusses the challenges facing foundations and offers a nine-part framework to suggest how an evaluation office can help foundations meet these challenges. The emphasis on capacity building is also of interest to organizations other than foundations. (SLD)

This report summarizes findings from a two-year study of Chicago's Excellence in Teaching Pilot, which was designed to drive instructional improvement by providing teachers with evidence-based feedback on their strengths and weaknesses. The pilot consisted of training and support for principals and teachers, principal observations of teaching…

A modified form of problem-based learning (PBL) with problems based on real workplace scenarios was trialled in a third year university class on Environmental Geology. Problems were developed in consultation with industry and based on their recent projects. These were then modified to allow for the shorter timeframe available, the less developed…

The current study will examine possible contextual effects relative to differences in reading comprehension performance in the state of Florida. While the Reading First (RF) Impact study examined such difference using a regression discontinuity design, the authors are primarily interested in other analytic methods that might answer different…

This document presents a five-step process for schools, districts, and states to use in monitoring accommodations for instruction and assessment. This document was designed to be a companion to the "Council of Chief State School Officers' Accommodations Manual: How to Select, Administer, and Evaluate Use of Accommodations for Instruction and…