This study examined the effects of marking method and rater experience on ESL (English as a Second Language) essay test scores and rater performance. Each of 31 novice and 29 experienced raters rated a sample of ESL essays both holistically and analytically. Essay scores were analysed using a multi-faceted Rasch model to compare test-takers'…

Intended to familiarize persons with the scoring standards and criteria used for the Georgia Basic Skills Writing Test, this scoring manual is in eight sections: (1) an introduction to the scoring dimensions and scale points; (2) definitions of the four scale points; (3) definitions of scoring dimensions and components (content and organization,…

Intended to familiarize persons with the scoring standards and criteria used for the 6th grade version of the Georgia Criterion-Referenced Writing Test, this scoring manual is in eight sections: (1) an introduction to the scoring dimensions and scale points; (2) definitions of the four scale points; (3) definitions of scoring dimensions and…

Intended to familiarize persons with the scoring standards and criteria used for the 8th grade version of the Georgia Criterion-Referenced Writing Test, this scoring manual is in eight sections: (1) an introduction to the scoring dimensions and scale points; (2) definitions of the four scale points; (3) definitions of scoring dimensions and…

Because there is no commonly accepted view of what makes for good writing, automated essay scoring (AES) ideally should be able to accommodate different theoretical positions, certainly at the level of state standards but also perhaps among teachers at the classroom level. This paper presents a practical approach and an interactive computer…

This study evaluated a "substantively driven" method for scoring NAEP writing assessments automatically. The study used variations of an existing commercial program, e-rater[R], to compare the performance of three approaches to automated essay scoring: a "brute-empirical" approach in which variables are selected and weighted solely according to…

Investigates whether training raters to interpret the scoring dimensions on a rubric would increase reliability. Compares two groups of kindergarten and first-grade teachers: one group with training, one without. Finds that training increases raters' abilities to reliably interpret scoring items. (SC)

This report identifies the methods and issues in the evaluation of student writing in classrooms and large scale assessments using an analytical scoring system from the Illinois Inventory of Educational Progress (IIEP). This guide is written for staff in state offices of education conducting large-scale writing assessments, and also for personnel…

Composition theory and research indicate that, whether they use a holistic or an analytic rubric, teachers should share their scoring guide with their students. Both students and teachers benefit in substantial ways when students draft and revise their written work in the context of the rubric their teachers will use to assess student writing.…

A follow-up study of two instruments for evaluating college writing was conducted. The experimental scale (E Scale) was developed in 1976 and revised for this study. The control scale (C Scale) was described in the literature in 1977. Ten English majors graded ten essays from diagnostic entrance exams. Both the E Scale and the C Scale were used,…

Four systems for scoring student essays were used to classify eleventh grade and undergraduate students according to writing ability. The reliabilities of the raters and the validities of the systems in classifying students were emphasized. Two analytic scoring systems--which assume that quality writing is characterized by the inclusion of certain…