Many academic tests (e.g. short-answer and multiple-choice) sample required knowledge with questions scoring 0 or 1 (dichotomous scoring). Few textbooks give useful guidance on the length of test needed to do this reliably. Posey's binomial error model of 1932 provides the best starting point, but allows neither for heterogeneity of question…

Nonlinear factor analysis is a tool commonly used by measurement specialists to identify both the presence and nature of multidimensionality in a set of test items, an important issue given that standard Item Response Theory models assume a unidimensional latent structure. Results from most factor-analytic algorithms include loading matrices,…

The literature on metacognition suggests that having students practice metacognitive monitoring consistently should lead to significant improvement over time. In this study, students practiced metacognitive monitoring through the course of a full semester. The authors then examined changes in monitoring accuracy, judgment bias, and their…

Choosing a strategy for controlling item exposure has become an integral part of test development for computerized adaptive testing (CAT). This study investigated the performance of six procedures for controlling item exposure in a series of simulated CATs under the generalized partial credit model. In addition to a no-exposure control baseline…

This article shows how set covering with item sampling (SCIS) methods can be used in the analysis and preanalysis of linear programming models for test assembly (LPTA). LPTA models can construct tests, fulfilling a set of constraints set by the test assembler. Sometimes, no solution to the LPTA model exists. The model is then said to be…

Based on a previously validated cognitive processing model of reading comprehension, this study experimentally examines potential generative components of text-based multiple-choice reading comprehension test questions. Previous research (Embretson & Wetzel, 1987; Gorin & Embretson, 2005; Sheehan & Ginther, 2001) shows text encoding and decision…

Item bias is a major threat to measurement validity. Methods for detecting differential item functioning (DIF) are now commonly used to identify potentially biased items. DIF detection methods for dichotomous items are well developed, but those for ordinal items are less well developed. In this article, the authors compare four methods for…

In international large-scale surveys, constructed response (CR) items are increasingly being used and multiple-choice (MC) items are being used less frequently. In this article the two item types will be compared in terms of any differences they have on national mean scores. TIMSS 1995 and TIMSS 1999 data have been used. Are there different…

While such recent legislation as the No Child Left Behind Act (NCLB) mandates monitoring the progress of all students--including English language learners (ELL)--the validity and fairness of assessment tools for such students remains a major concern for educational researchers and practitioners. The rapid increase in the number of ELL students…

A criterion-referenced computerized test is expressed as a statistical hypothesis problem. This admits that it can be studied by using the theory of optimal design. The power function of the statistical test is used as a criterion function when designing the test. A formal proof is provided showing that all items should have the same item…

Purpose: The purposes of this investigation were to examine the construct dimensionality and range of ability effectively measured by 28 assessment items obtained from 3 different patient-reported scales of communicative functioning, and to provide a demonstration of how the Rasch approach to measurement may contribute to the definition of latent…

The Nation's Report Card[TM] informs the public about the academic achievement of elementary and secondary students in the United States and its jurisdictions, including Puerto Rico. In 2003, a trial NAEP mathematics assessment was administered in Spanish to public school students at grades 4 and 8 in Puerto Rico. Based on preliminary analyses of…

This paper presents the assessment and item specifications for the National Assessment of Educational Progress (NAEP) 2009 mathematics assessment. "Chapter Two" contains descriptions of the five major content areas of mathematics (Number Properties and Operations, Measurement, Geometry, Data Analysis, Statistics, and Probability, and…

This report describes the results of two studies designed to evaluate the impact of self-reported mathematics course-taking on performance on SAT mathematics questions measuring new content (Algebra II). A supplemental table and figure are appended.

ACT college test publisher provided scores. On average, non-homeschoolers performed better than homeschoolers, by about two items, out of sixty items, on the ACT mathematics test that was analyzed. This result may be due to the different teaching/learning media used in teaching each of the two groups, to different teacher/student interaction, or…