ERIC Number: EJ792210
Record Type: Journal
Publication Date: 2008-Jan
Pages: 21
Abstractor: Author
ISBN: N/A
ISSN: ISSN-0895-7347
EISSN: N/A
Available Date: N/A
The Rating and Matching Item-Objective Alignment Methods
D'Agostino, Jerome V.; Welsh, Megan E.; Cimetta, Adriana D.; Falco, Lia D.; Smith, Shannon; VanWinkle, Waverely Hester; Powers, Sonya J.
Applied Measurement in Education, v21 n1 p1-21 Jan 2008
Central to the standards-based assessment validation process is an examination of the alignment between state standards and test items. Several alignment analysis systems have emerged recently, but most rely on either traditional rating or matching techniques. Little, if any, analyses have been reported on the degree of consistency between the two methods and on the item and objective characteristics that influence judges' decisions. We randomly assigned judges to either rate item-objective links or match items to objectives while reviewing the 2004 Arizona high school mathematics standards and assessment. Across items we found moderate convergence between methods, and we detected apparent reasons for divergently scored items. We also found that judges relied on item and objective content and intellectual skill features to render decisions. Based on our evidence, we contend that a thorough alignment analysis would involve judges using both rating and matching, while focusing on both content and intellectual skill. The findings have important implications for states when examining the alignment between their standards and assessments. (Contains 2 tables and 3 figures.)
Descriptors: Test Items, Student Evaluation, State Standards, Evaluation Methods, Secondary School Mathematics, Academic Standards, High School Students, Mathematics Teachers, Graduate Students, Scores, Correlation, Measurement Techniques, Error Patterns, Judges
Lawrence Erlbaum. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals/default.html
Publication Type: Journal Articles; Reports - Research
Education Level: High Schools
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Arizona
Grant or Contract Numbers: N/A
Author Affiliations: N/A