Several analyses of the construct validity of the fourth-grade, eighth-grade, and commencement-level English and Mathematics examinations of New York state were performed. The analyses present construct and differential construct elaboration both across tests and within tests. Results show strong relationships among different question types,…

As part of the test development process, this document provides the released mathematics items and student responses to some questions from the Michigan High School Proficiency Test (HSPT) in mathematics. This packet is designed for teachers and students who are interested in understanding how the open-ended items from the HSPT in Mathematics are…

This paper proposes an item selection algorithm that can be used to neutralize the effect of time limits in computer adaptive testing. The method is based on a statistical model for the response-time distributions of the test takers on the items in the pool that is updated each time a new item has been administered. Predictions from the model are…

A common practice in some certification fields (e.g., information technology) is to draw items from an item pool randomly and apply a common passing score, regardless of the items administered. Because these tests are commonly used, it is important to determine how accurate the pass/fail decisions are for such tests and whether fairly small,…

Judgmental standard setting methods, such as the W. H. Angoff (1971) method, use item performance estimates as the basis for determining the minimum passing score (MPS). Therefore, the accuracy of these item performance estimates is crucial to the validity of the resulting MPS. Recent researchers, (L. A. Shephard 1994; J. Impara, 1997) have called…

This study investigated whether equating accuracy improves with an anchor test that is more representative of its corresponding total test and whether such content effect depends on the particular equating method used. Scoring outcomes of a professional examination for a medical specialty were used. A total of 1,092 examinees took one form, and…

This paper provides both logical and empirical evidence to justify the use of an item mapping method for establishing passing scores for multiple-choice licensure and certification examinations. After describing the item-mapping standard setting process, the paper discusses the theoretical basis and rationale for this newly developed method and…

This study applied nonparametric bootstrapping to test null hypotheses for selected statistics (KR-20, difficulty, and discrimination) derived from a student-made test. The test, administered to 21 students enrolled in a graduate-level educational assessment class, contained 42 items, 33 of which were analyzed. Random permutations of the data…

Item scores that do not fit an assumed item response theory model may cause the latent trait value to be estimated inaccurately. For computerized adaptive tests (CAT) with dichotomous items, several person-fit statistics for detecting nonfitting item score patterns have been proposed. Both for paper-and-pencil (P&P) test and CATs, detection of…

The convention for selecting starting points (that is, initial items) on a computerized adaptive test (CAT) is to choose as starting points items of medium difficulty for all examinees. Selecting a starting point based on prior information about an individual's ability was first suggested many years ago, but has been believed unimportant provided…

This book explains writing effective multiple-choice test items and studying responses to items to evaluate and improve them, two topics that are very important in the development of many cognitive tests. The chapters are: (1) "Providing a Context for Multiple-Choice Testing"; (2) "Constructed-Response and Multiple-Choice Item Formats"; (3)…

This handbook explains how to do Rasch measurement. The emphasis is on practice, but theoretical explanations are also provided. The Forward contains an introduction to the topic of Rasch measurement. Chapters 2, 4, 5, and 7 use a small problem to illustrate the application of Rasch measurement in detail, and methodological issues are considered…

This document contains samples and explanations from Colorado's state reading and writing comprehension test for Grade 4, hereby released to the public (parents, teachers, and any other interested parties). It presents one reading comprehension item, the scoring guide for that item, and 6 different student responses, with the scores they received.…

This study investigated the practice of weighting a type of test item, such as constructed response, more than other types of items, such as selected response, to compute student scores for a mixed-item type of test. The study used data from statewide writing field tests in grades 3, 5, and 8 and considered two contexts, that in which a single…

Examines the operationalization of marital quality indices used as dependent variables and the functioning and construction of marital quality variables. Presents both semantic and empirical criteria to judge the development of a marital quality index and a Quality Marriage Index (QMI) based on the introduced criteria. (Author/RC)