From 2006 to 2008, Educational Testing Service (ETS) produced a series of reports titled "A Culture of Evidence," designed to capture a changing climate in higher education assessment. A decade later, colleges and universities already face new and different challenges resulting from societal, technological, and scientific influences.…

In the present offering we challenge the presumption that the educational testing of students provides objective information about such students. This presumption largely rests on an empiricist account of science. In light of mounting criticism, however, empiricist foundationalism has given way to a social epistemology. From this standpoint,…

Longitudinal change in the average Z scores for four groups of pupils sorted by quartiles was examined for its stability over three years. The data, collected from 1998 to 2009, was obtained from nine cohorts of Japanese junior high school pupils totaling 1,962 subjects. It showed illusionary declines among the mid-range pupils but improvements…

Items on test score scales located at and below the Proficient cut score define the content area knowledge and skills required to achieve proficiency. Alternately, examinees who perform at the Proficient level on a test can be expected to be able to demonstrate that they have mastered most of the knowledge and skills represented by the items at…

Recently, educational researchers and practitioners have turned to value-added models to evaluate teacher performance. Although value-added estimates depend on the assessment used to measure student achievement, the importance of outcome selection has received scant attention in the literature. Using data from a large, urban school district, I…

On a general level, the objective of diagnostic classifications models (DCMs) lies in a classification of individuals regarding multiple latent skills. In this article, the authors show that this objective can be achieved by multidimensional adaptive testing (MAT) as well. The authors discuss whether or not the restricted applicability of DCMs can…

In this study, we describe a framework for monitoring rater performance over time. We present several statistical indices to identify raters whose standards drift and explain how to use those indices operationally. To illustrate the use of the framework, we analyzed rating data from the 2002 Advanced Placement English Literature and Composition…

Although the Angoff procedure is among the most widely used standard setting procedures for tests comprising multiple-choice items, research has shown that subject matter experts have considerable difficulty accurately making the required judgments in the absence of examinee performance data. Some authors have viewed the need to provide…

In this article, we introduce a person-fit statistic called the hierarchy consistency index (HCI) to help detect misfitting item response vectors for tests developed and analyzed based on a cognitive model. The HCI ranges from -1.0 to 1.0, with values close to -1.0 indicating that students respond unexpectedly or differently from the responses…

Distance Education has globally become one of the important solutions for increasing admission into the universities, decongesting campuses and efficient utilization of time and space. To ensure the sustainability of the programmes' noble objectives calls for periodic re-evaluation of its modus operandi including the assessment of the perception…

The author offers some thoughts on commentator's reactions to the substance of the measures, particularly those about measuring teacher learning and change, based on the major uses of the measures, and because this is a significant challenge facing test development as an enterprise. If teacher learning results in more integrated knowledge or…

Two focus groups of test and measurement experts were held to explore the use of out-of-level testing for students with disabilities. The participants (n=17) included state and federal level assessment personnel, test company employees, and university professors. A content analysis of the narrative results indicated that there was no clear…

Placement exams are high-stakes assessments that determine many students' college trajectories. More than half of entering students at community colleges are placed into developmental education in at least one subject, based primarily on scores from these assessments, yet recent research fails to find evidence that placement into remediation…

In this article, the author echoes his co-author's and colleague's pleasure (Hill, this issue) at the thoughtfulness and far-ranging nature of the comments to their initial attempts at test validation for the mathematical knowledge for teaching (MKT) measures using the validity argument approach. Because of the large number of commentaries they…

In this study, special education teachers identified students with learning disabilities who were working on math skills usually taught two grades below the grade in which the student was enrolled. Each student (n=33) took two levels of the MAT/7 math computation test, an on-grade test, and an out-of-level test intended for students two grades…