The test review process used by the Buros Center for Testing is described as a series of 11 steps: (1) identifying tests to be reviewed, (2) obtaining tests and preparing test descriptions, (3) determining whether tests meet review criteria, (4) identifying appropriate reviewers, (5) selecting reviewers, (6) sending instructions and materials to…

In a provocative and influential paper, Jesse Rothstein (2010) finds that standard value added models (VAMs) suggest implausible future teacher effects on past student achievement, a finding that obviously cannot be viewed as causal. This is the basis of a falsification test (the Rothstein falsification test) that appears to indicate bias in VAM…

Most experts in the testing community have presumed that the $350 million promised by the U.S. Department of Education to support common assessments would promote those that made greater use of open-ended items capable of measuring higher-order critical-thinking skills. But as measurement experts consider the multitude of possibilities for an…

This paper outlines the use of item analysis to assist examiners in evaluating the quality and validity of their MCQ exam questions. The generation of item analysis, particularly discrimination index, has long been established practice in professional testing and credentialing organisations and some disciplines in tertiary education, but its use…

This article describes how assessments in PE Metrics were developed following six steps: (a) determining test blueprint, (b) writing assessment tasks and scoring rubrics, (c) establishing content validity, (d) piloting assessments, (e) conducting item analysis, and (f) modifying the assessments based on analysis and expert opinion. A task force,…

The proper use of psychological tests requires that the measurement instruments have adequate psychometric properties, such as reliability and validity, and that the professionals who use the instruments have the necessary expertise. In this article, we present the first review of tests published in Spain, carried out with an assessment model…

The purpose of this article is to define and clarify the process of instructional problem-solving using assessment data within action research (AR) and Response to Intervention (RtI). Similarities between AR and RtI are defined and compared. Lastly, specific resources and examples of the instructional problem-solving process of AR within…

The experience of the International Society of Addiction Medicine in setting up the first international certification of clinical knowledge is reported. The steps followed and the results of a psychometric analysis of the tests from the first 65 candidates are reported. Lessons learned in the first 5 years and challenges for the future are…

The one-parameter logistic model with ability-based guessing (1PL-AG) has been recently developed to account for effect of ability on guessing behavior in multiple-choice items. In this study, the authors developed algorithms for computerized classification testing under the 1PL-AG and conducted a series of simulations to evaluate their…

This supplement describes national adaptations made to the international version of the TIMSS 2011 background questionnaires. This information provides users with a guide to evaluate the availability of internationally comparable data for use in secondary analyses involving the TIMSS 2011 background variables. Background questionnaire adaptations…

It is commonly assumed that a significant item analysis (F2) provides an assurance that the treatment effect is generalizable to the population of items from which the items were drawn, which in turn implies that the effect is reasonably general across items. The latter implication is shown to be false, and it is argued that a new test of…

To rule out the need for a two-parameter item response theory (IRT) model during item analysis by Rasch models, it is important to check the Rasch model's assumption that all items have the same item discrimination. Biserial and polyserial correlation coefficients measuring the association between items and restscores are often used in an informal…

This article addresses the leadership preparation line of inquiry developed in the past decade by the University Council for Educational Administration/Learning and Teaching in Educational Leadership Special Interest Group Taskforce on Evaluating Leadership Preparation Programs, and it particularly addresses the series of survey instruments…

In this article, we highlight the need for a precisely defined construct in score-based validation and discuss the contribution of cognitive theories to accurately and comprehensively defining the construct. We propose a framework for integrating cognitively based theoretical and empirical evidence to specify and evaluate the construct. We apply…

Correlations between self-reported learning gains and direct, longitudinal measures that ostensibly correspond in content area are generally inadequate. This chapter clarifies that self-reported measures of learning are more properly used and interpreted as evidence of students' perceived learning and affective outcomes. In this context, the…