A method of estimating the parameters of the Rasch Model removing the effect of ramdom guessing is presented. The procedure is an application of the ARRG (Abilities Removing Random Guessing) model recently developed for two parameter latent trait models. Under the Rasch model ARRG provides for estimation of abilities, removing the effects of…

We focus on the problem of ignoring statistical independence. A binomial experiment is used to determine whether judges could match, based on looks alone, dogs to their owners. The experimental design introduces dependencies such that the probability of a given judge correctly matching a dog and an owner changes from trial to trial. We show how…

A modified beta binomial model is presented for use in analyzing random guessing multiple choice tests and taste tests. Detection probabilities for each item are distributed beta across the population subjects. Properties for the observable distribution of correct responses are derived. Two concepts of true score estimates are presented.…

The purpose of this paper is to derive explicit estimates of the parameters of a latent structure model when the skills represented by the test items are hierarchically related. Two special cases are described, which may be used as an approximation to the Dayton and Macready model and also provide initial estimates in an iterative estimation…

Two basic issues are discussed that involve the Carver model of the relationship between test score and knowledge. The first of these concerns different types of tests which have different probabilities of obtaining the correct answer due to guessing. A derivation is shown which places different test types "on-a-par"--meaning that two tests are…

A model is presented for item responses when different subjects use different strategies, but only responses--not choice of strategy--can be observed. Substantive theory is used to differentiate the likelihoods of response vectors under a fixed set of strategies, and response probabilities are modeled via item parameters for each strategy. (TJH)

Macready and Dayton introduced two probabilistic models for mastery assessment based on an idealistic all-or-none conception of mastery. Alternatively, an application of latent trait theory to mastery testing is proposed (a three parameter logistic model) as a more plausible model for test theory. (Author/CTM)

On well-constructed multiple-choice tests, the most serious threat to measurement is not variation in item discrimination, but the guessing behavior that may be adopted by some students. Ways of ameliorating the effects of guessing are discussed, especially for problems in latent trait models. A new item response model, including an item parameter…

This document contains three papers from the Methodology Project of the Center for the Study of Evaluation. Methods for characterizing test accuracy are reported in the first two papers. "Bounds on the K Out of N Reliability of a Test, and an Exact Test for Hierarchically Related Items" describes and illustrates how an extension of a…

The Answer-Until-Correct (AUC) procedure has been proposed in order to increase the reliability of multiple-choice items. A model for examinees' behavior when they must respond to each item until they answer it correctly is presented. An expression for the reliability of AUC items, as a function of the characteristics of the item and the scoring…

In multiple-choice items the response probability on an item may be viewed as the result of two distinct latent processes--a cognitive process to solve the problem, and another random process that leads to the choice of a certain alternative (the process of giving the actual response). An incomplete latent class model is formulated that describes…

Some stochastic approximation procedures are considered in relation to the problem of choosing a sequence of test questions to accurately estimate a given examinee's standing on a psychological dimension. Illustrations are given evaluating certain procedures in a specific context. (Author/CK)

This pamphlet reprints three papers and an invited discussion of them, read at a Division 5 Symposium at the 1975 American Psychological Association Convention. The first paper describes a Bayesian tailored testing process and shows how it demonstrates the importance of using test items with high discrimination, low guessing probability, and a…