Faced with selecting a geometry curriculum for our preservice elementary teacher mathematics course, we used a mixed-methods study to investigate the effectiveness, with respect to student achievement and student perception, of three reform-oriented curricula. ANCOVA results indicate students using one of the curricula scored significantly higher…

This study evaluated a "substantively driven" method for scoring NAEP writing assessments automatically. The study used variations of an existing commercial program, e-rater[R], to compare the performance of three approaches to automated essay scoring: a "brute-empirical" approach in which variables are selected and weighted solely according to…

This guide to ERRATA, a set of HyperCard-based tools for response analysis and error diagnosis in language testing, is intended as a user manual and general reference and designed to be used with the software (not included here). It has three parts. The first is a brief survey of computational techniques available for dealing with student test…

This manual is designed to assist both special and regular educators with mastering the skills for developing quality teacher-made tests consistent with content-oriented instruction. The manual presents tips for constructing both objective and subjective, supply and select test questions--namely, short answer, essay, fill in the blanks or…

This procedures manual for the North Carolina Test Development Program details the tasks involved in test development. The basic steps are described in an introductory section; they are (1) to specify the institutional objectives of the curriculum and ensure their instructional validity; and (2) to specify the characteristics, such as the purpose…

The Mantel-Haenszel procedure (MH) is a practical, inexpensive, and powerful way to detect test items that function differently in two groups of examinees. MH is a natural outgrowth of previously suggested chi square methods, and it is also related to methods based on item response theory. The study of items that function differently for two…

In latent trait theory the latent space, or space of the hypothetical construct, is usually represented by some unidimensional or multi-dimensional continuum of real numbers. Like the latent space, the item response can either be treated as a discrete variable or as a continuous variable. Latent trait theory relates the item response to the latent…

The nonparametric approach to test theory discussed here has its roots in the early work of Guttman, Lazarsfeld, and Meredith; and more recently in the work of Cliff and in Tatsuoka and Tatsuoka. Mokken's extensive treatment of this subject concentrated on defining, constructing, and testing unidimensional scales, based on responses to dichotomous…

This paper provides an up-to-date review of the relationship between item response theory (IRT) and (nonlinear) common factor theory and draws out of this relationship some implications for current and future research in IRT. Nonlinear common factor analysis yields a natural embodiment of the weak principle of local independence in appropriate…

In connection with the development of item pools for the Navy's experimental Computerized Adaptive Testing (CAT) system, items in five different content areas (General Science, Paragraph Composition, Word Knowledge, Math Knowledge, and Arithmetic Reasoning) have been calibrated using item response theory (IRT) methodology. Operational Armed…

The determination of appropriate cut scores is a critical step in the development of licensing and certification examinations. Passing point methodologies based on the estimation of item difficulties are underlain by the estimation of the probability of a correct response to items by a hypothetically minimally competent candidate. The Angoff…

This study was designed to investigate the empirical consequences of building tests for which the size of the difference in average scores between Black and White examinees is made to be as large and as small as possible within certain constraints. Test forms for two item types were developed that maximized or minimized the differences in average…

The problem of sex bias in mathematics word problems is discussed, with references to the appropriate literature. Word problems are assessed via cognitive science analysis of word problem solving. It has been suggested that five basic semantic relations are adequate to classify nearly all story problems, namely, change, combine, compare, vary, and…

The Test Analysis Package (TAP) consists of three microcomputer programs that edit, score, summarize, and analyze student test data. Features of the TAP system include: (1) an editor that allows creation of a student test data file without the use of a separate word processor; (2) a scoring mechanism that can score a test data file of 999 students…

A solution to reduce the likelihood of one examinee copying another's answers on large scale tests that require all examinees to answer the same set of questions is to use multiple test forms that differ in terms of item ordering. This study was conducted to determine whether varying the sequence in which blocks of items were presented to…