Discusses effects of test question order on student performance. Addresses (1) differences in the distribution of scores on each form of the examination; (2) effects on the validity of individual examination items; and (3) effects on the reliability of the examination instrument. Concludes that distribution of examination scores may be influenced…

The technique of nonmetric multidimensional scaling (MDS) was applied to real item response data obtained from a multiple-choice achievement test of unknown dimensionality. The goal was to classify the 50 items into the various subtests from which they were drawn originally, the latter being unknown to the investigator. Issues addressed in the…

For over 50 years, the overwhelming weight of evidence has been that subjects are able to make use of partial information when responding to multiple-choice items. The subject chooses the alternative which has given rise to the lowest mismatch, except that if this minimum mismatch is larger than some threshold, the question is left unanswered.…

This report investigates an area of uncertainty in using the Angoff method for setting standards, namely whether or not a judge's conceptualizations of borderline group performance are realistic. Ratings are usually made with reference to the performance of this hypothetical group, therefore the Angoff method's success is dependent on this point.…

One means of learning about the processes operating in a multiple choice test is to include some test items, called nonsense items, which have no correct answer. This paper compares two versions of a mathematical model of test performance to interpret test data that includes both genuine and nonsense items. One formula is based on the usual…

The purpose of this study is to examine an option-weighting method as it affects pass-fail decisions in formative and summative evaluation of student achievement for instructional units, certification, advancement, licensure, admissions, placement, and selection. A database was constructed using high school achievement test data where a…

A solution to reduce the likelihood of one examinee copying another's answers on large scale tests that require all examinees to answer the same set of questions is to use multiple test forms that differ in terms of item ordering. This study was conducted to determine whether varying the sequence in which blocks of items were presented to…

In this paper 45 item-writing rules for multiple-choice tests presented in textbooks on educational measurement in a previous study are identified. The current study presents a quantitative review of the literature with respect to the empirical and theoretical evaluation of these principles of item-writing. Fifty-six studies that addressed at…

The objectives for this study were to: (1) develop a valid, reliable measure of test-wiseness with equivalent forms for use with students in the health sciences; and (2) determine the level of test-wiseness of entering medical students. The test-wiseness areas included in this study were: similar options, umbrella term, item give-away, convergence…

Despite the popularity of both multiple-choice (MC) and true-false (TF) items, most investigations comparing the two formats have done so to determine the optimum number of choices to be given to students within a given time period. The purpose of this investigation was to compare the reliabilities and the validities of both formats when the items…

The purposes of the present study were to ascertain whether or not two particular formatting characteristics influenced seventh grade student performance on math tests. One research question focused on the effects of decimal positioning in multiple choice options for computational items. In a second question, labeling format was compared with…

This study examined characteristics of multiple-choice reading comprehension tasks suspected of influencing their difficulty, through administration of the California Achievement Tests to 158 deaf college students. Problem components evaluated included manifest content, psychologically salient features, and processing demands. Variation in item…

This paper deals with test fairness regarding a test consisting of two parts: (1) a "common" section, taken by all students; and (2) a "variable" section, in which some students may answer a different set of questions from other students. For example, a test taken by several thousand students each year contains a common multiple-choice portion and…

The comparative effects of various sequences of test items were examined for over 900 graduate students enrolled in an educational research course at The University of Alabama, Tuscaloosa. experiment, which was conducted a total of four times using four separate tests, presented three different arrangements of 50 multiple-choice items: (1)…

This paper reports a multi-method approach for examining the cognitive level of multiple-choice items used in a medical pathology course at a large midwestern medical school. Analysis of the standard item analysis data and think-out-loud reports of a sample of students completing a 66 item examination were used to test assumptions related to the…