Study was designed to provide some indication as to whether the gain in test reliability from confidence scoring results from relevant or irrelevant sources. If the increase in reliability is the result of a gambling response style, it is conceivable that validity could actually decrease even though reliability is increased. (Authors/CB)

An attempt is made to explore the use of subjective probabilities in the analysis of item data, especially criterion-referenced item data. Two assumptions are implicit: (1) one wants to obtain a maximum amount of information with respect to an item using a minimum number of subjects; and (2) once the item is validated, it may well be administered…

Binary, probability, and ordinal scoring procedures for multiple-choice items were examined. In two situations, it was found that both the probability and ordinal scoring systems were more reliable than the binary scoring method. (Author/CTM)

A potentially valuable measure of overconfidence on probabilistic multiple-choice tests was evaluated. The measure of overconfidence was based on probabilistic responses to nonsense items embedded in a vocabulary test. The test was administered under both confidence response and conventional choice response directions to 208 undergraduate…

The effects of incentive conditions on the results of a confidence test were investigated. Two hundred thirty high school subjects were administered a very difficult confidence scored test under two conditions: 1) that the test would count heavily on their grades (incentive condition) and 2) that the test was for research purposes and would not be…

Probabilistic testing involves having the examinee assign probabilities to each of the options of a multiple-choice item. These probabilities reflect the student's perception of the correctness of each option. What is presented in the paper is a rationale for probability testing, the current theoretical and empirical findings, and some suggested…

This paper examines confidence testing, and reasons for using confidence tests. Different scoring systems are studied in order to clarify the meaning of significance of the weights which subjects assign to confidence scored tests. (DLG)

A comparison of reliability and validity was made for three testing procedures: 1) responding conventionally to Verbal Ability and Mathematical Reasoning tests; 2) using a confidence weighting response procedure with the same tests; and 3) using the elimination response method. The experimental testing procedures were not psychometrically superior…

Confidence testing has been used in varying forms over the past 40 years as a method for increasing the amount of information available from objective test items. This paper traces the development of the procedure from Hevner's beginning method up to the various methods in use today and describes both the testing procedures and scoring methods…

Various aspects of Confidence Weighting are examined. Variant of Confidence Weighting, its effect on test reliability, and the validity of Confidence Weighting are discussed. (DG)

Technical improvements, which may be made in the reliability and validity of tests through confidence scores, are discussed. However, studies indicate that subjects do not handle their confidence uniformly. (MS)