ERIC Number: EJ1193113
Record Type: Journal
Publication Date: 2018
Pages: 15
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0895-7347
EISSN: N/A
Available Date: N/A
Applying a Thurstonian, Two-Stage Method in the Standardized Assessment of Writing
McGrane, Joshua Aaron; Humphry, Stephen Mark; Heldsinger, Sandra
Applied Measurement in Education, v31 n4 p297-311 2018
National standardized assessment programs have increasingly included extended written performances, amplifying the need for reliable, valid, and efficient methods of assessment. This article examines a two-stage method using comparative judgments and calibrated exemplars as a complement and alternative to existing methods of assessing writing. Written performances were taken from Australia's National Assessment Program-Literacy and Numeracy, which included both narrative and persuasive performances from students aged 8 to 15. In Stage 1, assessors performed comparative judgments on 160 performances to form a scale of 36 calibrated exemplars. These comparative judgments showed a very high level of reliability and concurrent validity. In Stage 2, assessors scored 2,380 new performances by matching them to the most similar calibrated exemplar. These matching judgments showed a generally high level of reliability and concurrent validity and were reasonably efficient after a familiarization period. Further research is suggested to enhance Stage 2 by simplifying the exemplar scale and scaffolding it with detailed descriptors. Overall, the findings support the use of the method in standardized writing assessment and its application to various learning areas.
Descriptors: Standardized Tests, Foreign Countries, Writing Tests, Writing Evaluation, National Competency Tests, Interrater Reliability, Validity, Elementary School Students, Secondary School Students, Grade 3, Grade 5, Grade 7, Grade 9, Efficiency, Evaluation Methods
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Australia
Grant or Contract Numbers: N/A
Author Affiliations: N/A