Two-stage and multistage test designs provide a way of roughly adapting item difficulty to test taker ability. This study incorporated testlets (bundles of items) into two-stage and multistage designs, and compared the precision of the ability estimates derived from these designs with those derived from a standard computerized adaptive test (CAT)…

A methodological approach that emerged during the design of task-specific research rubrics to code large sets of open-ended survey data fills the void in scholarship about developing rubrics for research purposes. A brief rationale for using this method rather than other, often-used, data analysis methods is provided, with a description of the…

A continuing and vexing problem associated with survey instrument development is the creation of items, initially, that correlate favorably a posteriori with constructs being measured. This study tests the use of symbolic-logic matrices developed by D. G. Coleman (1979) in creating factorially "pure" statistically discrete constructs in…

This study is a critical evaluation of the roles for coding and scoring of missing responses to multiple-choice items in educational tests. The focus is on tests in which the test-takers have little or no motivation; in such tests omitting and not reaching (as classified by the currently adopted operational rules) is quite frequent. Data from the…

This fastback provides information about what tests can and cannot do, how they are constructed, and how they are used and misused. It offers suggestions about how best to interpret test results. The chapters are: (1) "A Test on Testing"; (2) "Basic Considerations"; (3) "Standardized Tests"; (4) "Performance Tests"; (5) "Interpreting Test Scores";…

In today's educational setting, assessment results weigh heavily in determining what students should know and be able to do. In addition, because assessment scores are often tied to accountability systems that affect both teaching and learning, they influence what is taught in the classroom. The changes in the use of assessments underscore the…

This paper applies specific information item selection using a method developed by T. Davey and M. Fan (2000) to a multiple-choice passage-based reading test that is being developed for computer administration. Data used to calibrate the multidimensional item parameters for the simulation study consisted of item responses from randomly equivalent…

A series of studies was conducted to develop and validate a scale to study racism in its newly adapted, ambivalent form. The scale was designed to measure the separate positively correlated hostile and benevolent components of racism. The first two studies, which involved 39 and 73 college students, established the questions that could be used for…

Data from a 50-item translated test used for certification were used to assess the percentage and type of agreement between the Mantel-Haenszel (MH) and Differential Functioning of Items and Tests (DFIT) techniques for the detection of differential item functioning (DIF). The DFIT procedure flagged 10 of 30 items as exhibiting significant DIF…

With the advent of computer-based testing (CBT) and the need to increase the number of items available in computer adaptive test pools, the idea of item variants was conceived. An item variant can be defined as an item with content based on an existing item to a greater or lesser degree. Item variants were first proposed as a way to enhance test…

The recent National Research Council (NRC) report, "Knowing What Students Know: The Science and Design of Educational Assessment," suggests that it is time to rethink the basic assumptions underlying assessment of students and the use of measurement data to enhance teaching and learning. This essay draws on arguments developed in the NRC report to…

A method based on 0-1 linear programming (LP) is presented to stratify an item pool optimally for use in "alpha"-stratified adaptive testing. Because the 0-1 LP model belongs to the subclass of models with a network-flow structure, efficient solutions are possible. The method is applied to a previous item pool from the computerized…

The three-step procedure for judging social studies exercises developed for the NAEP is described for use by schools and communities in considering performance standards for their own students. (Author/KM) Aspect of National Assessment (NAEP) dealt with in this document: Procedures (Utilization).

Suggestions for adapting NAEP procedures for local use are based on the assumptions that instructional decisions can be improved through systematic gathering of data and that every student is not tested. (Author/KM) Aspect of National Assessment (NAEP) dealt with in this document: Procedures (Utilization).

A discussion of how quiz questions can be very effective teachers. They may condition students to study in particular ways, to attend to some types of information and ignore others, and they may determine how the student processes information. (Author)