ERIC Number: EJ741762
Record Type: Journal
Publication Date: 2006
Pages: 25
Abstractor: Author
ISBN: N/A
ISSN: ISSN-0022-0655
EISSN: N/A
Available Date: N/A
Evaluating DETECT Classification Accuracy and Consistency when Data Display Complex Structure
Gierl, Mark J.; Leighton, Jacqueline P.; Tan, Xuan
Journal of Educational Measurement, v43 n3 p265-289 Fall 2006
DETECT, the acronym for Dimensionality Evaluation To Enumerate Contributing Traits, is an innovative and relatively new nonparametric dimensionality assessment procedure used to identify mutually exclusive, dimensionally homogeneous clusters of items using a genetic algorithm ( Zhang & Stout, 1999). Because the clusters of items are mutually exclusive, this procedure is most useful when the data display approximate simple structure. In many testing situations, however, data display a complex multidimensional structure. The purpose of the current study was to evaluate DETECT item classification accuracy and consistency when the data display different degrees of complex structure using both simulated and real data. Three variables were manipulated in the simulation study: The percentage of items displaying complex structure (10%, 30%, and 50%), the correlation between dimensions (0.00, 0.30, 0.60, 0.75, and 0.90), and the sample size (500, 1,000, and 1,500). The results from the simulation study reveal that DETECT can accurately and consistently cluster items according to their true underlying dimension when as many as 30% of the items display complex structure, if the correlation between dimensions is less than or equal to 0.75 and the sample size is at least 1,000 examinees. If 50 of the items display complex structure, then the correlation between dimensions should be less than or equal to 0.60 and the sample size be, at least, 1,000 examinees. When the correlation between dimensions is 0.90, DETECT does not work well with any complex dimensional structure or sample size. Implications for practice and directions for future research are discussed.
Descriptors: Program Evaluation, Cluster Grouping, Evaluation Methods, Multivariate Analysis, Test Items, Simulation, Predictor Variables, Correlation, Sample Size, Item Analysis, Nonparametric Statistics, Multidimensional Scaling
Blackwell Publishing. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8599; Fax: 781-388-8232; e-mail: customerservices@blackwellpublishing.com; Web site: http://www.blackwellpublishing.com/jnl_default.asp.
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A