A case study illustrates the ways in which local reactions to a program evaluation directly threaten its validity, and shows that not all participants in a qualitative evaluation study understand or defer to the commitments of democratic evaluation. Ethical concerns in program evaluation are considered. (SLD)

Outlines problems involved in information retrieval evaluation and suggests a more user-centered interpretive approach. A longitudinal study that examines evaluation through the information-seeking behavior of 22 case studies of actual users is described; and relevance judgments, effectiveness ratios, and analysis of document failures are…

Practices in needs assessment are reviewed since 1981, the year the federal government changed the role it plays in promoting needs assessment in the public sector. The review is based on reports published to 1993 and indicates that needs assessment is flourishing in a variety of settings. (SLD)

Considers the competing demands placed on evaluators in relation to the service (characteristics of the intervention), the sample (people participating), and methodology or research design. Explores compromises that may be required between scientific ideals and real-world limitations, and assesses implications for obtaining meaningful results in…

A number of small countries face similar problems while organising external assessment of higher education institutions: how to select proper experts, how to prepare them and maintain? How to avoid bias or interest conflict between experts who evaluate the institution and the institution being evaluated? How to reduce the financial costs of…

This handbook is designed to help educators understand program evaluation processes so that they will be able to carry out evaluation tasks and talk knowledgeably to evaluators about what they want to find out from a program evaluation. The book reviews the purposes and uses of evaluation to help the reader identify the scope and direction of the…

Quality has become a priority issue for all concerned with early childhood care and education services. Starting from the premise that quality is a relative and dynamic concept based on values and beliefs, this collection advocates that the process of establishing definitions of quality should involve a range of stakeholder groups. The chapters…

The extent to which staff appraisal has been implemented at Britain's public sector further education (FE) institutions and the characteristics, benefits, and disadvantages of existing appraisal systems were examined in a survey of 233 FE institutions (36% of all institutions in the public FE sector). Of the FE institutions surveyed, 76% had one…

For the past 20 years, the University of Hawaii Community Colleges have had a formal process for the review of educational programs, requiring that each program undergo a systematic review at least once every 5 years. The result tended to be a lengthy document that chronicled the history of the previous 5 years and was far more descriptive than…

Indicators of mathematics and science education progress are important to policymakers and the public in order to evaluate the quality of education. Recent reports have evaluated existing indicators and suggested core aspects of mathematics and science education that an indicator system should evaluate. This note discusses two fundamental aspects…

An examination of the policies and processes used to evaluate teaching in colleges and universities over the past several decades begins by identifying six key features of the research literature: (1) no formal definition of effective teaching has been generated to guide instrument development, data collection, and interpretation; (2) the purpose…

The evaluation team of the Lincoln Public Schools (Nebraska) used the multi-attribution utility technology (MAUT) approach to prioritize potential evaluation projects. The priorities were used to allocate resources to the district's most important projects, and to eliminate or scale down less important projects. The problem was caused initially…

In October 2002, the Institute of Education Sciences (IES) contracted with Mathematica Policy Research, Inc. (MRP) to help identify issues pertinent to the evaluation of Title I and to propose feasible evaluation design strategies. This design effort took its lead from two sources: (1) the Title I Independent Review Panel (IRP); and (2) a more…

Standardized testing plays an increasingly important role in the lives of today's students and educators. The U.S. No Child Left Behind Act (NCLB) requires assessment in math and literacy in grades 3-8 and 10 and, as of 2007-08, in science once in grades 3-5, 6-9, and 10-12. Based on National Center for Education Statistics enrollment projections,…

The author responds to Pinnell et al's (1996) call for caution in using cost-benefit analysis in measuring educational reform success. It is argued that reform already is being sold to state legislators and educators in cost-benefit terms and that research has been inappropriately used in the selling process. (GR)