Writing multiple-choice test items to measure student learning in higher education is a challenge. Based on extensive scholarly research and experience, the author describes various item formats, offers guidelines for creating these items, and provides many examples of both good and bad test items. He also suggests some shortcuts for developing…

The ability to reason about numbers in relative terms is essential for quantitative literacy, which is necessary for studying academic disciplines and for critical citizenship. However, the ability to reason with proportions is known to be difficult to learn and to take a long time to develop. To determine how well higher education applicants can…

Digital information literacy (DIL)--generally defined as the ability to obtain, understand, evaluate, and use information in a variety of digital technology contexts--is a critically important skill deemed necessary for success in higher education as well as in the global networked economy. To determine whether college graduates possess the…

This study employed an eye-tracking technique to examine students' visual attention when solving a multiple-choice science problem. Six university students participated in a problem-solving task to predict occurrences of landslide hazards from four images representing four combinations of four factors. Participants' responses and visual attention…

Describes a study of undergraduates that investigated the effects of providing access to a case library of related stories while solving ill-structured problems. Describes experimental, comparable, and control group treatments and discusses results of multiple-choice tests and short answer questions. (Author/LRW)

Twenty-four students who scored 650 or more on the Scholastic Aptitude Test Mathematics test (SAT-M) were asked to think aloud while solving 13 mathematics items in either multiple-choice or free-response format. Strategies students used to solve the items were classified as either algorithmic or insightful. Data analyses indicated that items in…

A new item type was developed, incorporating features of "ill-structured" problems in a multiple-choice format. The problems are similar to previously developed scientific thinking tasks in requiring the examinee to go beyond the information provided; they resemble a variant of the logical reasoning item type, but demand somewhat more structuring…

The reliability and validity of a procedure to measure diagnostic-reasoning and problem-solving skills taught in predoctoral orthodontic education were studied using 68 second year dental students. The procedure includes stimulus material and 33 multiple-choice items. It is a feasible way of assessing problem-solving skills in dentistry education…

Multiple-choice tests such as the Force Concept Inventory (FCI) provide useful instruments to probe the distribution of student difficulties on a large scale. However, traditional analysis often relies solely on scores (number of students giving the correct answer). This ignores what can be significant and important information: the distribution…

Within the next two decades, calculators will be standard tools in mathematics classrooms. They should be used regularly but carefully in classroom instruction and testing to allow use of realistic data and exploration of mathematical concepts, but not to allow finding the answer without understanding the question. (MSE)

A study was conducted to determine whether competently reasoning clinicians (clinical instructors in medical instruction) could identify reasons competently reasoning students would eliminate distractors, and explain how students would reason to select the keyed response when solving multiple-choice items. The think-aloud protocols of clinicians…

To help determine the role that test instrument formats play in evaluation, two parallel examinations were given to 227 second-year medical students. The tests were based on information presented in a medical case history. One required students to generate their own problem lists (the generate group); the other required the students to select…

Investigates how second language intake was affected both by awareness and by the conditions under which a problem-solving task was performed. Spanish conditional sentences were presented to learners through five different degrees of explicitness. Intake was measured through a multiple-choice recognition test administered immediately after the…

Explains how multiple choice test items can be devised to measure higher-order learning, including engineering problem solving. Discusses the value and information provided in item analysis procedures with machine-scored tests. Suggests elements to consider in test design. (ML)

Six measurement strategies are suggested for assessing different kinds or aspects of competencies: (1) teaching tests; (2) behavior samples; (3) teaching exercises; (4) classroom interaction simulations; (5) projected problem exercises; and (6) paper-and-pencil exercises. (MJB)