The University of Hawaii developed an assessment and testing program for qualifying internal medicine residents in basic procedural skills: an observation period, a multiple-choice examination, and a slide-identification examination. The mean scores of both examinations were analyzed to assess whether curriculum changes were effective. (Author/MLW)

Extended matching sets questions (EMSQs) are a form of multiple-choice question (MCQ) consisting of a stem (the question or scenario) with an extended number of possible answers. Although there is no consensus on their absolute format, for the purpose of this paper a multiple-choice question with ten or more alternative answers is considered to be…

The present study examined how a reader's subject knowledge, the analogy versus nonanalogy difference in text type, and type of test (written recall, sentence completion, and multiple choice) affect first language (L1) and second language (L2) reading comprehension. There were three participant groups: (a) 53 native Costa Ricans enrolled in…

The quality of mathematics multiple choice items and their susceptibility to test wiseness were examined. Test wiseness was defined as "a subject's capacity to utilize the characteristics and formats of the test and/or test taking situation to receive a high score." The study used results of the Graduate Record Examinations Aptitude Test (GRE) and…

This document contains three papers from the Methodology Project of the Center for the Study of Evaluation. Methods for characterizing test accuracy are reported in the first two papers. "Bounds on the K Out of N Reliability of a Test, and an Exact Test for Hierarchically Related Items" describes and illustrates how an extension of a…

Online practice tests and quizzes are commonly available to higher education students. The extent of student use of such study tools and the relationship of use to achievement, however, have not been fully investigated. One hundred twelve college students were encouraged to use optional online quizzes in preparation for proctored examinations.…

In latent trait models the standard procedure for handling the problem caused by guessing on multiple choice tests is to estimate a parameter which is intended to measure the "guessingness" inherent in an item. Birnbaum's three parameter model, which handles guessing in this manner, ignores individual differences in guessing tendency. This paper…

Scores representing number of items right and self-perceptions were analyzed for a nonquantitative examination that was assembled into three forms. Multivariate ANCOVA revealed no significant effects for the cognitive measure. However, significant sex and sex x order effects were found for perceptions scores not parallel to those reported…

The appropriate analysis of students' responses to an assessment is an essential step in improving the quality of the assessment itself as well as staff teaching and student learning. Many academics are unfamiliar with the formal processes used to analyze assessment results; the standard statistical methods associated with analyzing the validity…

This digest describes some basics of the construction of multiple-choice tests. As a rule, the test maker should strive for test item stems (introductory questions or incomplete statements at the beginning of each item that are followed by the options) that are clear and parsimonious, answers that are unequivocal and chosen by the students who do…

Omitted items cannot properly be treated as wrong when estimating ability and item parameters. A convenient method for utilizing the information provided by omissions is presented. Theoretical and empirical justifications are presented for the estimates obtained by the new method. (Author)

This paper focuses on the Non-Equivalent Groups with Anchor Test (NEAT) design for test equating and on two classes of observed--score equating (OSE) methods--chain equating (CE) and poststratification equating (PSE). These two classes of methods reflect two distinctly different ways of using the information provided by the anchor test for…

This paper deals with test fairness regarding a test consisting of two parts: (1) a "common" section, taken by all students; and (2) a "variable" section, in which some students may answer a different set of questions from other students. For example, a test taken by several thousand students each year contains a common multiple-choice portion and…

Difficulty and discrimination ability were compared between multiple choice and short answer items in midterm and final examinations for the internal medicine course at Louisiana State University School of Dentistry. The examinations were administered to 67 sophomore dental students in that course. Additionally, the impact of the source of the…

A study investigated the effectiveness of the Rasch procedure in measuring response appropriateness, especially for the detection of cheating on multiple-choice language tests. The report gives background information on appropriateness measurement and its potential uses, reviews recent research on cheating and its detection, and describes three…