A probabilistic approach is introduced to classify and diagnose erroneous rules of operation resulting from a variety of misconceptions ("bugs") in a procedural domain of arithmetic. The model is contrasted with the deterministic approach which has commonly been used in the field of artificial intelligence, and the advantage of treating the…

Using conventional mental test theory, item parameters of an aptitude or achievement test vary with each group of examinees, and as such are somewhat limited in their use and interpretation. Within the last 25 years, measurement models have emerged in which item parameters are considered to be invariant. Generically referred to as latent trait…

In the methods and approaches developed for estimating the operating characteristics of the discrete item responses, the maximum likelihood estimate of the examinee based upon the "Old Test" has an important role. When Old Test does not provide a sufficient amount of test information for the upper and lower part of the ability interval,…

A classical problem in mastery testing is the choice of passing score and test length so that the mastery decisions are optimal. This problem has been addressed several times from a variety of viewpoints. In this paper, the usual indifference zone approach is adopted, with a new criterion for optimizing the passing score. Specifically,…

The rule space model permits measurement of cognitive skill acquisition and error diagnosis. Further discussion introduces Bayesian hypothesis testing and bug distribution. An illustration involves an artificial intelligence approach to testing fractions and arithmetic. (Author/GDC)

The progress of modern mental test theory depends very much on the techniques of maximum likelihood estimation, and many popular applications make use of likelihoods induced by logistic item response models. While, in reality, item responses are nonreplicate within a single examinee and the logistic models are only ideal, practitioners make…

Appropriateness measurement is a general approach to the problem caused by multiple choice tests failing to measure accurately the ability of atypical examinees. The conceptual framework of appropriateness measurement is presented, and several statistical indices of the appropriateness of a multiple choice test for an examinee are noted. A series…

This study introduced a probabilistic model utilizing item response theory (IRT) for dealing with a variety of misconceptions. The model can be used for evaluating the transition behavior of error types, advancement of learning stages, or the stability and persistence of particular misconceptions. Moreover, it apparently can be used for relating…

In the context of pass/fail decisions, using the Bock multi-nominal latent trait model for moderate-length tests does not produce decisions that differ substantially from those based on the raw scores. The Bock decisions appear to relate less strongly to outside criteria than those based on the raw scores. (Author/JAZ)

Advantages and disadvantages of joint maximum likelihood, marginal maximum likelihood, and Bayesian methods of parameter estimation in item response theory are discussed and compared. (Author)

A newly introduced approach, rule space, can represent large numbers of erroneous rules of arithmetic operations quantitatively and can predict the likelihood of each erroneous rule. The new model challenges the credibility of the traditional right-or-wrong scoring procedure. (Author/PN)

The rule space model permits measurement of cognitive skill acquisition, diagnosis of cognitive errors, and detection of the strengths and weaknesses of knowledge possessed by individuals. Two ways to classify an individual into his or her most plausible latent state of knowledge include: (1) hypothesis testing--Bayes' decision rules for minimum…

This paper introduces a probabilistic model that is capable of diagnosing and classifying cognitive errors in a general problem-solving domain. Item response theory is used to deal with the variability of response errors. Responses from a 38 item fraction addition test given to 595 junior high school students are used to illustrate the model.…