ERIC Number: EJ903604
Record Type: Journal
Publication Date: 2010
Pages: 21
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1075-2935
EISSN: N/A
Available Date: N/A
Playing with the Stakes: A Consideration of an Aspect of the Social Context of a Gatekeeping Writing Assessment
Baker, Beverly A.
Assessing Writing, v15 n3 p133-153 2010
In high-stakes writing assessments, rater training in the use of a rating scale does not eliminate variability in grade attribution. This realisation has been accompanied by research that explores possible sources of rater variability, such as rater background or rating scale type. However, there has been little consideration thus far of variability in rater behaviour that may be brought about by the socio-political context of the testing situation. In addition, studies of rater behaviour are undertaken in either research conditions or authentic high-stakes testing conditions, but never both. This report is a "stakes study" in the context of a writing test for teacher certification in Quebec. The research objective was to see if raters grade differently when the consequences to test takers are changed but if all else remains the same (text, rater, rating scale and rater training). Scores given by the raters in the "high-stakes" (authentic testing) and "low-stakes" (research) conditions were compared. Raters were also questioned immediately after their final grading session to allow them to discuss any perceived differences in their rating behaviour in the two conditions. It was hypothesised that raters might be more severe in their grading if there were no stakes involved for the test takers. This case study, while exploratory and small in scale, demonstrates the potential in including the social context as a factor in future research on variability in rater scoring behaviour. (Contains 4 tables and 1 figure.)
Descriptors: Foreign Countries, Writing Evaluation, Writing Tests, Testing, High Stakes Tests, Rating Scales, Grading, Teacher Certification, Context Effect, Scores, Interviews, Case Studies, Comparative Analysis, Interrater Reliability, Testing Problems, Evaluation Problems
Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail: usjcs@elsevier.com; Web site: http://www.elsevier.com.bibliotheek.ehb.be
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Canada
Grant or Contract Numbers: N/A
Author Affiliations: N/A