NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED424272
Record Type: Non-Journal
Publication Date: 1997-Dec
Pages: 107
Abstractor: N/A
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
A Study of Equating in NAEP. NAEP Validity Studies.
Hedges, Larry V.; Vevea, Jack L.
This study investigates the amount of uncertainty added to National Assessment of Educational Progress (NAEP) estimates by equating error under both ideal and less than ideal circumstances. Data from past administrations are used to guide simulations of various equating designs and error due to equating is estimated empirically. The design includes a variety of factors that might affect the accuracy of equating, with the levels of each factor based roughly on operational values in the NAEP 1992 and 1994 reading and 1992 mathematics assessments. Specific factors investigated were the number of items in the scale, the proportion of items in the scale taken by each student, the proportion of items in each administration that are common to each, the proportion of each item type in each scale, the proportion of each item type among common items used for equating, the scale linking strategy (item response theory invariance, common item, or multiple group item response theory linking), and the change in ability from data wave 1 to wave 2. Common item scale linking performed very well, even under circumstances that were less than ideal, including slight to moderate multidimensionality. Mean bias was estimated to be more than about 0.01 to 0.02 standard deviations. However, in nonideal conditions there were biases in the extreme quantiles of the ability distribution, even with no population shifts. These biases were several times as large as the mean bias and could be large enough to create problems in tracking low performance and the means of low performing groups over several waves of assessment. When both waves of data can be scaled together, multiple group item response theory methods provided very accurate scale linking with virtually no bias. (Contains 29 tables, 42 figures, and 9 references.) (SLD)
Publication Type: Numerical/Quantitative Data; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: National Center for Education Statistics (ED), Washington, DC.
Authoring Institution: American Institutes for Research in the Behavioral Sciences, Palo Alto, CA.
Identifiers - Assessments and Surveys: National Assessment of Educational Progress
Grant or Contract Numbers: N/A
Author Affiliations: N/A