This report provides a description of the design for the National Assessment of Educational Progress (NAEP) 1994 Trial State Assessment in Reading and gives an overview of the steps involved in the implementation of the program from the planning stages through to the analysis and reporting of data. The report does not provide the results of the…

This study reports the results of an investigation into the accuracy and efficacy of item calibration schemes used by two commercially available personal computer programs, BILOG and MicroCAT, when used to calibrate a test that is currently used in higher education. A calibration of 1,000 randomly selected students' responses to a 72-question math…

A method is proposed to assess the importance of differential item functioning (DIF) by estimating the largest possible fraction of the population in which DIF does not occur, or equivalently, the smallest possible portion of the population in which DIF may occur. The approach is based on latent class (C. C. Clogg, 1981) or mixture concepts, and…

The usual assessment of speededness for rights-only scored tests does not account for the possibility that examinees respond in a random or patterned fashion to the items at the end of the test as the time limit approaches. This study represented an attempt to determine if Sections 2 and 3 of the Test of English as a Foreign Language (TOEFL) are…

The National Association of Boards of Pharmacy (NABP) manages the National Association of Boards of Pharmacy Licensure Examination (NABPLEX). A new Scope of Pharmacy Practice study was completed in 1994, and representatives of NABP and the National Association of Pharmacy Regulatory Authorities of Canada met in 1995 to discuss the possibility of…

Traditional approaches to the investigation of the objectivity of ratings for constructed-response items are based on classical test theory, which is item-dependent and sample-dependent. Item response theory overcomes this drawback by decomposing item difficulties into genuine difficulties and rater severity. In so doing, objectivity of ability…

Clear descriptions of job expertise are required to support applications and improvements in personnel training and job performance. This report describes a practical approach to task analysis that integrates the issues, content, and methods of cognitive science and personnel psychology. Cognitively oriented task analysis employs a breadth, then…

Recent advances in psychometrics and computer technology encourage the development of model-based methods of individualized testing on a microcomputer, where each examinee receives short tests and where the number of pretest items that can be administered is severely restricted. On-line (i.e., data is collected on operational equipment) methods…

In the periodic testing environment associated with conventional paper-and-pencil tests, the frequency with which items are seen by test-takers is tightly controlled in advance of testing by policies that regulate both the reuse of test forms and the frequency with which candidates may take the test. In the continuous testing environment…

The design of vocabulary tests, particularly for English as a Second Language, is discussed. The discussion is intended to help language teachers with little or no knowledge of testing gain a better understanding of vocabulary testing. First, a set of principles for guiding the writing of vocabulary tests are outlined, presented in the form of…

This digest discusses the advantages and disadvantages of using item banks, and it provides useful information for those who are considering implementing an item banking project in their school districts. The primary advantage of item banking is in test development. Using an item response theory method, such as the Rasch model, items from multiple…

A simulation study was performed to determine whether a group's average percent correct in a content domain could be accurately estimated for groups taking a single test form and not the entire domain of items. Six Item Response Theory (IRT) -based domain score estimation methods were evaluated, under conditions of few items per content area per…

Although computerized adaptive tests acquire their efficiency by successively selecting items that provide optimal measurement at each examinee's estimated level of ability, operational testing programs will typically consider additional factors in item selection. In practice, items are generally selected with regard to at least three, often…

In this paper it is shown that differential item functioning can be evaluated using the Lagrange multiplier test or C. R. Rao's efficient score test. The test is presented in the framework of a number of item response theory (IRT) models such as the Rasch model, the one-parameter logistic model, the two-parameter logistic model, the generalized…

Many textbooks in educational measurement and classroom assessment have chapters devoted to specific item formats. There may be attempts to relate one item format to another, but the chapters and item formats are largely seem as distinct entities with only loose and uncertain connections. This paper synthesizes these discussions. An item format…