ERIC Number: ED600861
Record Type: Non-Journal
Publication Date: 2017
Pages: 48
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-
EISSN: N/A
Available Date: N/A
On the Complexity of Item Response Theory Models
Bonifay, Wes; Cai, Li
Grantee Submission
Complexity in item response theory (IRT) has traditionally been quantified by simply counting the number of freely estimated parameters in the model. However, complexity is also contingent upon the functional form of the model. The information-theoretic principle of minimum description length provides a novel method of investigating complexity by considering the inherent propensity of a model to fit well to any possible data. We examine four popular IRT models--exploratory item factor analytic, bifactor, DINA, and DINO--with different functional forms but the same number of free parameters. In comparison, a simpler (unidimensional 3PL) model was specified such that it had 1 more free parameter than the previous models. All five models were then fit to 1,000 data sets that were randomly and uniformly sampled from the complete data space and each model was assessed using global and item-level fit and diagnostic measures. The findings revealed that the factor analytic and bifactor models possess excessive flexibility and therefore a strong tendency to fit any possible data. The unidimensional 3PL model displayed minimal fitting propensity, despite the fact that it included an additional free parameter. The DINA and DINO models did not demonstrate a proclivity to fit any possible data, but they did fit well to meaningfully different data patterns. These findings suggest that applied researchers and psychometricians should consider functional form--and not goodness-of-fit alone--when selecting and applying an IRT model. [This paper was published in "Multivariate Behavioral Research" v52 n4 p465-484 2017.]
Descriptors: Item Response Theory, Difficulty Level, Goodness of Fit, Factor Analysis, Models, Data, Classification
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: National Center for Education Research (ED)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305D140046
Author Affiliations: N/A