Multiple choice results are inherently probabilistic outcomes, as correct responses reflect a combination of knowledge and guessing, while incorrect responses additionally reflect blunder, a confidently committed mistake. To objectively resolve knowledge from responses in an MC test structure, we evaluated probabilistic models that explicitly…

This article examines whether the way that PISA models item outcomes in mathematics affects the validity of its country rankings. As an alternative to PISA methodology a two-parameter model is applied to PISA mathematics item data from Canada and Finland for the year 2012. In the estimation procedure item difficulty and dispersion parameters are…

Item development is a time- and resource-intensive process. Automatic item generation integrates cognitive modeling with computer technology to systematically generate test items. To date, however, items generated using cognitive modeling procedures have received limited use in operational testing situations. As a result, the psychometric…

Computerized adaptive testing (CAT) is a mode of testing which enables more efficient and accurate recovery of one or more latent traits. Traditionally, CAT is built upon Item Response Theory (IRT) models that assume unidimensionality. However, the problem of how to build CAT upon latent class models (LCM) has not been investigated until recently,…

A model and method for developing a valid and reliable paper-and-pencil test for evaluating the technical knowledge of baseball coaches in the National Coaching Certification Program (NCCP) in Canada are discussed. Development of a general theoretical and operational model for the evaluation of NCCP coaches in all sports are also studied. The…

In preparation for a study of essay questions and other forms of open-ended exercises in the California Golden State Examination for biology, the functioning of open-ended biology items in another examination was explored. The Golden State Examination program offers honors credit to students who wish to qualify for admission to programs in…