ERIC Number: ED603425
Record Type: Non-Journal
Publication Date: 2019
Pages: 40
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-
EISSN: N/A
Available Date: N/A
Item Calibration Methods with Multiple Sub-Scale Multistage Testing
Wang, Chun; Chen, Ping; Jiang, Shengyu
Grantee Submission
Many large-scale educational surveys have moved from linear form design to multistage testing (MST) design. One advantage of MST is that it can provide more accurate latent trait [theta] estimates using fewer items than required by linear tests. However, MST generates incomplete response data by design; hence questions remain as to how to calibrate items using the incomplete data from MST design. Further complication arises when there are multiple correlated subscales per test, and when items from different subscales need to be calibrated according to their respective score reporting metric. The current calibration-per-subscale method produced biased item parameters, and there is no available method for resolving the challenge. Deriving from the missing data principle, we showed when calibrating all items together, the Rubin's (1976) ignorability assumption is satisfied such that the traditional single-group calibration is sufficient. When calibrating items per subscale, we proposed a simple modification to the current calibration-per-subscale method that helps reinstate the missing-at-random assumption and therefore corrects for the estimation bias that is otherwise existent. Three mainstream calibration methods are discussed in the context of MST, they are the marginal maximum likelihood estimation (MML), the expectation maximization (EM) method, and the fixed parameter calibration (FPC). An extensive simulation study is conducted and a real data example from NAEP is analyzed to provide convincing empirical evidence. [This report was published in "Journal of Educational Measurement."]
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED); National Science Foundation (NSF)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305D170042; R305D160010; SES165932
Author Affiliations: N/A