ERIC Number: EJ838616
Record Type: Journal
Publication Date: 2008-Feb
Pages: 53
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1540-2525
EISSN: N/A
Available Date: N/A
Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees' Cognitive Skills in Algebra on the SAT
Gierl, Mark J.; Wang, Changjiang; Zhou, Jiawen
Journal of Technology, Learning, and Assessment, v6 n6 Feb 2008
The purpose of this study is to apply the attribute hierarchy method (AHM) to a sample of SAT algebra items administered in March 2005. The AHM is a psychometric method for classifying examinees' test item responses into a set of structured attribute patterns associated with different components from a cognitive model of task performance. An attribute is a description of the procedural or declarative knowledge needed to perform a task. These attributes form a hierarchy of cognitive skills that represent a cognitive model of task performance. The study was conducted in two steps. In step 1, a cognitive model was developed by having content specialists, first, review the SAT algebra items, identify their salient attributes, and order the item-based attributes into a hierarchy. Then, the cognitive model was validated by having a sample of students think aloud as they solved each item. In step 2, psychometric analyses were conducted on the SAT algebra cognitive model by evaluating the model-data fit between the expected response patterns generated by the cognitive model and the observed response patterns produced from a random sample of 5000 examinees who wrote the items. Attribute probabilities were also computed for this random sample of examinees so diagnostic inferences about their attribute-level performances could be made. We conclude the study by describing key limitations, highlighting challenges inherent to the development and analysis of cognitive diagnostic assessments, and proposing directions for future research. (Contains 1 endnote, 4 tables and 11 figures.)
Descriptors: Test Items, Protocol Analysis, Psychometrics, Algebra, Inferences, Cognitive Ability, Thinking Skills, Models, Evaluation Research, Problem Solving, Skill Analysis, Task Analysis, Cognitive Measurement, Cognitive Tests
Technology and Assessment Study Collaborative. 332 Campion Hall, Boston College, Chestnut Hill, MA 02467. Tel: 617-552-4521; Fax: 617-552-8419; e-mail: jtal_editor@bc.edu; Web site: http://escholarship.bc.edu/jtla
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: SAT (College Admission Test)
Grant or Contract Numbers: N/A
Author Affiliations: N/A