Rigorous comparison of the reliability coefficients of several tests or measurement procedures requires a sampling theory for the coefficients. This paper summarizes the important aspects of the sampling theory for Cronbach's (1951) coefficient alpha--a widely used internal consistency coefficient. This theory enables researchers to test a…

On a multiple-choice test in which each item has k alternative responses, the test taker is permitted to choose any subset which he believes contains the one correct answer. A scoring system is devised. (Author/CTM)

The purposes of the study were to develop a measure of overconfidence on probabilistic tests, to assess the measurement characteristics of such a measure, and to investigate the relationship of overconfidence on tests to knowledge and to risk-taking propensity. (Author/BB)

Describes confidence testing methods (confidence weighting, probabilistic marking, multiple alternative selection) as alternative to computer-based, multiple choice tests and explains potential benefits (increased reliability, improved examinee evaluation of alternatives, extended diagnostic information and remediation prescriptions, happier…

Focuses on a unique measurement procedure which allows students to indicate when and the extent to which they guess on an examination. Suggests that such a procedure increases the accuracy of information derived from objective tests. (MH)

An investigation is conducted which presents extensive Monte Carlo results which indicate the conditions under which a procedure using the F distribution can be used to study the robustness of the confidence interval procedures for small samples. A review of the literature is presented. Procedure uses a binary data matrix. Results indicate that…

Confidence weighting (CW) tends to improve the reliability of easy tests; the Coombs-type multiple-response (MR) option tends to improve the reliability of hard tests. It was hypothesized that, on a test of moderate difficulty, offering both the CW and MR response options would improve reliability more than either alone. Twenty-four subjects took…

Study was designed to provide some indication as to whether the gain in test reliability from confidence scoring results from relevant or irrelevant sources. If the increase in reliability is the result of a gambling response style, it is conceivable that validity could actually decrease even though reliability is increased. (Authors/CB)

The characteristics of three threshold loss agreement indices which reflect the agreement or consistency in assignment to mastery-nonmastery status are reviewed. These are proportion of agreement, coefficient kappa, and modified kappa. (Author/MT)

An attempt is made to explore the use of subjective probabilities in the analysis of item data, especially criterion-referenced item data. Two assumptions are implicit: (1) one wants to obtain a maximum amount of information with respect to an item using a minimum number of subjects; and (2) once the item is validated, it may well be administered…

Binary, probability, and ordinal scoring procedures for multiple-choice items were examined. In two situations, it was found that both the probability and ordinal scoring systems were more reliable than the binary scoring method. (Author/CTM)

The relationship between students' scores on a verbal meaning test and their degrees of confidence in item responses was investigated. Subjects were black undergraduate students and they were administered a verbal meaning test by following a confidence testing procedure. (Author/BJG)

A potentially valuable measure of overconfidence on probabilistic multiple-choice tests was evaluated. The measure of overconfidence was based on probabilistic responses to nonsense items embedded in a vocabulary test. The test was administered under both confidence response and conventional choice response directions to 208 undergraduate…

The effects of incentive conditions on the results of a confidence test were investigated. Two hundred thirty high school subjects were administered a very difficult confidence scored test under two conditions: 1) that the test would count heavily on their grades (incentive condition) and 2) that the test was for research purposes and would not be…

Defines confidence testing and describes several formats which permit student selection and weighting of test items thereby enhancing student-teacher participation in the testing process. (MH)