Multiple choice results are inherently probabilistic outcomes, as correct responses reflect a combination of knowledge and guessing, while incorrect responses additionally reflect blunder, a confidently committed mistake. To objectively resolve knowledge from responses in an MC test structure, we evaluated probabilistic models that explicitly…

Game-based assessment (GBA) is a specific use of educational games that employs game activities to elicit evidence for educationally valuable skills and knowledge. While this approach can provide individualized and diagnostic information about students, the design and development of assessment mechanics for a GBA is a nontrivial task. In this…

Using the capabilities of expert knowledge structures, the researcher prepared test questions on the university calculus topic of "finding the area by integration." The quiz is divided into two types of multiple choice items (one out of four and one out of many). After the calculus course was taught and tested, the results revealed that…

To compute the probability of having a disease, given a positive test result, is a standard probability problem. The sensitivity and specificity of the test must be given and the prevalence of the disease. We ask how a test-maker might determine the tradeoff between sensitivity and specificity. Adding hypothetical costs for detecting or failing to…

This paper provides a history of ETS's role in developing assessment instruments and psychometric procedures for measuring change in large-scale national assessments funded by the Longitudinal Studies branch of the National Center for Education Statistics. It documents the innovations developed during more than 30 years of working with…

Several Bayesian approaches to the simultaneous estimation of the means of k binomial populations are discussed. This has particular applicability to criterion-referenced or mastery testing. (Author/JKS)

Topics concerning latent trait theory are addressed: (1) dimensionality of latent space, local independence, and item characteristic curves; (2) models--equations, parameter estimation, testing assumptions, and goodness of fit, (3) applications test developments, item bias, tailored testing and equating; and (4) advantages over classical…

Latent trait theory supposes that, in testing situations, examinee performance on a test can be predicted (or explained) by defining examinee characteristics, referred to as traits, estimating scores for examinees on these traits and using the scores to predict or explain test performance (Lord and Novick, 1968). In view of the breakthroughs in…

This chapter should not only acquaint the reader with the present state of the art on Criterion-Referenced (CR) measurement but also suggest possible directions for further inquiry. The goal of the first part of this chapter is to deal with the definitional dilemma of CR measurement by proceeding from the more traditional view of CR measurement to…