Discusses contemporary controversies in evaluation that are likely to be of continuing importance in the near future, Including: (1) the nature and role of substantive theory in evaluation; (2) the importance of advances in methods; and (3) the meaning of "success as a profession." Asserts the necessity of positioning evaluation as part of…

Describes and critiques medical school rankings by "U.S. News & World Report" on methodologic and conceptual grounds, arguing that the annual evaluation falls short in both areas. Presents three alternative categories of program quality indicators. Concludes that the rankings have no practical value and fail to meet standards of…

Research indicated that students often graduate from colleges and universities without encountering critical thinking skills in undergraduate coursework. The first national report raising concerns over the declining quality of education stemmed from the National Commission on Excellence in Education (1983), and findings based on disappointing test…

A report from the School Choice Demonstration Project examines issues concerning the funding formula used for the Milwaukee Parental Choice Program (MPCP). It finds that the program generates a net saving to taxpayers in Wisconsin but imposes a significant fiscal burden on taxpayers in Milwaukee. However, these findings depend significantly on how…

Some students are less accurately measured by typical reading tests than other students. By asking teachers to identify students whose performance on state reading tests would likely underestimate their reading skills, this study sought to learn about characteristics of less accurately measured students while also evaluating how well teachers can…

Treatment integrity (also referred to as "treatment fidelity," "intervention integrity," and "procedural reliability") is an important methodological concerning both research and practice because treatment integrity data are essential to making valid conclusions regarding treatment outcomes. Despite its relationship to validity, treatment…

The utility of value-added estimates of teachers' effects on student test scores depends on whether they can distinguish between high- and low-productivity teachers and predict future teacher performance. This article studies the year-to-year variability in value-added measures for elementary and middle school mathematics teachers from five large…

Purpose: The purpose of this paper is to investigate a possible favourable response pattern on scaled forms used as a means of evaluating training courses. Design/methodology/approach: In part one of the study evaluation forms were examined referring to courses were completed by 879 students attending 15 university level courses and 531 students…

This is a critical review of methodological issues in the evaluation of adult literacy education programs in the United States. It addresses the key research questions: What are the appropriate methods for evaluating these programs under given circumstances. It identifies 15 evaluation studies that are representative of a range of adult literacy…

This brief describes estimation and measurement issues relevant to estimating the quality of instruction in the context of a cumulative model of learning. It also discusses implications for the use of value-added estimates in personnel and compensation matters. The discussion highlights the importance of accounting for student differences and the…

This support document accompanies the NCVER report, "A Review of Indigenous Employment Programs," hereafter the "Report" [ED499667]. It contains explanatory notes on evaluation methodology, plus additional details on developments in labor market assistance and reviews of the literature relating to the evaluation of such…

The constraints of conducting evaluations in real-world settings often necessitate the implementation of less than ideal designs. Unfortunately, the standard method for estimating the precision of a result (i.e., confidence intervals [CI]) cannot be used for evaluative conclusions that are derived from multiple indicators, measures, and data…

Authentic assessment is a growing alternative to conventional testing. This research-to-practice article describes a framework for implementing authentic assessment. The R-E-A-L framework shows how roles, equipment, assessment tools, and location can be incorporated into early childhood practices.

On a general level, the objective of diagnostic classifications models (DCMs) lies in a classification of individuals regarding multiple latent skills. In this article, the authors show that this objective can be achieved by multidimensional adaptive testing (MAT) as well. The authors discuss whether or not the restricted applicability of DCMs can…

This paper is a polemical discussion of assessment in teacher education. Working from the proposition that assessment serves a number of important purposes for a range of stakeholders (students, employers, quality assurance agencies, government), it argues that there is considerable potential for conflict between the different purposes. In an age…