The standard error of measurement usefully provides confidence limits for scores in a given test, but is it possible to quantify the reliability of a test with just a single number that allows comparison of tests of different format? Reliability coefficients do not do this, being dependent on the spread of examinee attainment. Better in this…

There have been many attempts to formulate a procedure for extracting information from incorrect responses to multiple choice items, i.e., the assessment of partial knowledge. The results of these attempts can be described as inconsistent at best. It is hypothesized that these inconsistencies arise from three methodological problems: the…

The effects of luck and misinformation on ability of multiple-choice test scores to estimate examinee ability were investigated. Two measures of examinee ability were defined. Misinformation was shown to have little effect on ability of raw scores and a substantial effect on ability of corrected-for-guessing scores to estimate examinee ability.…

It has been demonstrated that corrected-for-guessing scores will be superior to number-right scores in providing estimates of examinee standing on the trait measured by a multiple-choice test, if it can be assumed that examinees can and will comply with the appropriate directions. The purpose of the present study was to test the validity of that…

One formulation of confidence scoring requires the examinee to indicate as a number his personal probability of the correctness of each alternative in a multiple-choice test. For this formulation, a linear transformation of the logarithm of the correct response is maximized if the examinee reports accurately his personal probability. To equate…

Seven formulas for scoring test items with two options (true-false or multiple choice with only two choices) were investigated. Several conditions, such as varying directions for guessing and whether testees had prior knowledge of the proportions of false items on the test were also investigated. (Author/JKS)

These studies in test adequacy focus on two problems: procedures for estimating reliability, and techniques for identifying ineffective distractors. Fourteen papers are presented on recent advances in measuring achievement (a response to Molenaar); "an extension of the Dirichlet-multinomial model that allows true score and guessing to be…

Current Scoring practices for multiple-choice tests are rooted in early Associationist Theory and are based on a two-step procedure: (1) right answers counted as ones and wrong answers are zeros, and (2) number of right answers form a total-correct score. The author contends that if either step is invalid, the use of the general linear model (GLM)…

A novel scoring procedure was investigated in order to obtain scores from a conventional multiple-choice test that would be free of the guessing component or contain a known guessing component even though examinees were permitted to guess at will. Scores computed with the experimental procedure are based not only on the number of items answered…

Corrected-for-guessing scores on multiple-choice tests depend upon the ability and willingness of examinees to guess when they have some basis for answering, and to avoid guessing when they have no basis. The present study determined the extent to which college students were able and willing to comply with formula-scoring directions. (Author/CTM)

During 1969 the International Association for the Evaluation of Educational Achievement began a series of cross-cultural studies to investigate the workings of multiple-choice achievement tests and student guessing behaviors. Empirical models to correct for guessing are discussed in terms of test item difficulty, number of response choices,…

A specially designed answer format was used for three tests in a college level agriculture class of 19 students to record responses to three things about each item: (1) the student's choice of the best answer; (2) the degree of certainty with which the answer was chosen; and (3) all the answer choices which the student was certain were incorrect.…

This handbook presents instructions for implementing a confidence testing program in technical training situations, identification of possible areas of application, techniques for evaluating confidence information, advantages and disadvantages of confidence testing, time considerations, and problem areas. Complete instructions for "Pick-One" and…

This paper examines confidence testing, and reasons for using confidence tests. Different scoring systems are studied in order to clarify the meaning of significance of the weights which subjects assign to confidence scored tests. (DLG)