This paper provides a history of ETS's role in developing assessment instruments and psychometric procedures for measuring change in large-scale national assessments funded by the Longitudinal Studies branch of the National Center for Education Statistics. It documents the innovations developed during more than 30 years of working with…

This digest introduces ways of responding to the call for criterion-referenced information using Bayes' Theorem, a method that was coupled with criterion-referenced testing in the early 1970s (see R. Hambleton and M. Novick, 1973). To illustrate Bayes' Theorem, an example is given in which the goal is to classify an examinee as being a master or…

Educators need accurate assessments of preschool cognitive growth to guide curriculum design, evaluation, and timely modification of their instructional programs. But available tests do not provide content breadth or growth sensitivity over brief intervals. This article details evidence for a multiform, multiscale test criterion-referenced to…

Presents a procedure, based on a Bayesian updating of the item information, for locating on the latent trait scale the scores or responses of items that follow the three-parameter logistic and monotone partial credit models. Applications are provided in terms of selecting items or score categories for criterion-referenced interpretation of mapping…

Several Bayesian approaches to the simultaneous estimation of the means of k binomial populations are discussed. This has particular applicability to criterion-referenced or mastery testing. (Author/JKS)

A Bayesian framework for making mastery/nonmastery decisions based on multivariate test data is described. Overall, mastery is granted if the posterion expected loss associated with such action is smaller than the one incurred by denying mastery. (Author/JKS)

Approaches to the determination of cut-scores have used threshold, normal ogive, linear and discrete utility functions. These approaches are examined by investigating conditions on the posterior, likelihood and utility functions required for setting cut-scores in a Bayesian approach. (Author/JKS)

Concern about passing those examinees who should pass, and retaining those who need remedial work, is one problem related to criterion-referenced testing. This paper deals with one aspect of that problem. When determining how many items to include on a criterion-referenced test, practitioners must resolve various non-statistical issues before a…

Item mappings are widely used in educational assessment for applications such as test administration (through test form assembly and computer assisted testing) and for criterion-referenced (CR) interpretation of test scores or scale anchoring. Item mappings are also used to construct ordered item booklets in the CTB/McGraw Hill Bookmark standard…

By noting that a Rasch or two parameter logistic (2PL) item belongs to the exponential family of random variables and that the probability density function (pdf) of the correct response (X=1) and the incorrect response (X=0) are symmetric with respect to the vertical line at the item location, it is shown that the conjugate prior for ability is…

Since the primary purpose of classical testing is to rank order examinees consistently, the absolute value of the true score has been relatively unimportant. However, the major purpose of criterion referenced testing is to estimate the true capabilities of examinees to perform specific tasks. Hence, the problems of true score determination assume…

Describes a procedure, called a full Bayesian procedure, for making decisions in connection with criterion-referenced measurements. The procedure uses continuous utility functions instead of a dichotomized utility structure and combines the posterior distribution for a certain person with utility functions for "advance" and "retain" decisions…

Bayes Theorem leads to three indexes of item effectiveness: 1) probability that an examinee knows the content given that the correct response was selected; 2) probability that an examinee does not know that content given that an incorrect response was selected; and 3) probability of making a correct decision about the examinee's knowledge given…

In this paper, an attempt has been made to synthesize some of the current thinking in the area of criterion-referenced testing as well as to provide the beginning of an integration of theory and method for such testing. (Editor)

Formulas are presented for decision reliability and for classification validity for mastery/nonmastery decisions based on criterion referenced tests. Two item parameters are used: the probability of a master answering an item correctly, and the probability of a nonmaster answering an item incorrectly. The theory explores the relationships of…