Ultimately, measures in research, testing, assessment and evaluation are used, or have implications, for ranking, intervention, feedback, decision-making or policy purposes. Explicit recognition of this fact brings the often-ignored and sometimes maligned concept of consequences to the fore. Given that measures have personal and social…

Daniel Koretz is the Henry Lee Shattuck Professor of Education at the Harvard Graduate School of Education. His research focuses on educational assessment and policy, particularly the effects of high-stakes testing on educational practice and the validity of score gains. He is the author of "Measuring Up: What Educational Testing Really Tells…

Haertel's argument is that one must "expand the scope of test validation to include indirect testing effects" because these effects are often the "rationale for the entire testing program." The author strongly agrees that this is essential. However, he maintains that Haertel's argument does not go far enough and that there are two additional…

Considering consequences in the evaluation of validity is not new although it is still debated by Paul E. Newton and others. The argument-based approach to validity entails an interpretative argument that explicitly identifies the proposed interpretations and uses of test scores and a validity argument that provides a structure for evaluating the…

This article is a commentary on Australia's involvement in the Programme for International Student Assessment (PISA) tests. It provides a rationale for Australia's participation in the PISA programme, the influences of PISA involvement on education policies and practices, and considerations and implications for school leaders and education…

Well before the examinations grade crisis of 2012, Michael Gove had set out clear intentions for reforming public examinations. Though he claimed to be improving examinations and assessment by replicating practices that took place in high-performing countries and thus improving the ability of the UK economy to "compete", this…

Over the past decade, international and national education agencies have begun to emphasize the improvement of the quality (rather than quantity) of education in developing countries. This trend has been paralleled by a significant increase in the use of educational assessments as a way to measure gains and losses in quality of learning. As…

Psychometricians continue to introduce new approaches to setting cut scores for educational assessments in an attempt to improve on current methods. In this paper we describe the Item-Descriptor (ID) Matching method, a method based on IRT item mapping. In ID Matching, test content area experts match items (i.e., their judgments about the knowledge…

It is argued that publishers of achievement tests, especially those who publish tests intended for use in many parts of the United States, are for the most part not in a position to obtain any decent evidence about the consequences of the uses that are made of their tests. What responsibilities and actions publishers can reasonably be expected to…

The use of item banks and item response theory has resulted in new ways to misinterpret and misuse tests through customized, yet standardized, achievement test batteries. The new test batteries create the possibility of serious misunderstandings based on the idea that any subset of items from the pool with a proper range of difficulties will…

The potential use of criterion referenced tests (CRT) and norm referenced tests (NRT) in the same testing program is discussed. The advantages and disadvantages of each are listed, and the best solution, a merging, is proposed. To merge CRTs and NRTs into an overall testing program, meaningful and useful to all levels, consideration must be given…

Specific recommendations are made concerning the circumstances under which the benefits of out-of-level testing outweigh the problems associated with it. Topics explored are: various methods for deciding when a set of test scores is invalid and the utility of these methods for local evaluators, the accuracy of vertical scaling, and the usefulness…

A defense of the System of Multicultural Pluralistic Assessment (SOMPA) in response to previous articles in this issue of this journal is presented by the test developer. The defense is detailed and addresses each previous article. (JKS)

Issues in designing valid tests for the National Assessment of Educational Progress (NAEP) are discussed. Test scores are often provided without any information on the nature of the tasks represented. Because test domains are defined by individual item writers, the generalizability between tests and items is suspect. While typical content…

The idea that America was being harmed because its schools were not keeping up with those in other advanced nations emerged after Sputnik in 1957, took a firm hold on education policy when "A Nation at Risk" appeared in 1983, and continues today. Policy makers justify this concern by pointing to evidence showing that, for individuals…