The literature on Automated Essay Scoring (AES) systems has provided useful validation frameworks for any assessment that includes AES scoring. Furthermore, evidence for the scoring fidelity of AES systems is accumulating. Yet questions remain when appraising the scoring performance of AES systems. These questions include: (a) which essays are…

This study examined the effects of marking method and rater experience on ESL (English as a Second Language) essay test scores and rater performance. Each of 31 novice and 29 experienced raters rated a sample of ESL essays both holistically and analytically. Essay scores were analysed using a multi-faceted Rasch model to compare test-takers'…

A procedure for creating a numerical index which can be used to score and rank summaries of prose passages is described. The quality of a summary is assumed to be a function of two variables: the number of important ideas mentioned, and the number of words used. Scoring involves multiplying two ratios: a power ratio and an efficiency ratio. The…

This paper presents a scientific rationale for deciding the number of points to use on a grading scale in any given assessment situation. The rationale is applied to two common methods of assessment (multiple-choice and essay tests) and an example of a composite assessment. (Author/SJL)

The essay on the writing section of the SAT will be scored using a holistic approach. In holistic scoring, a piece of writing is considered as a total piece of work--the whole of which is greater than the sum of its parts.

Using data from the SCRE Criterion Test composition papers, the author tests the hypothesis that the bulk of inter-marker unreliability is caused by inter-marker inconsistency--which is not correctable statistically. He suggests that a shift to "consensus" standards will realize greater improvements than statistical standardizing alone.…

Because there is no commonly accepted view of what makes for good writing, automated essay scoring (AES) ideally should be able to accommodate different theoretical positions, certainly at the level of state standards but also perhaps among teachers at the classroom level. This paper presents a practical approach and an interactive computer…

This paper deals with test fairness regarding a test consisting of two parts: (1) a "common" section, taken by all students; and (2) a "variable" section, in which some students may answer a different set of questions from other students. For example, a test taken by several thousand students each year contains a common multiple-choice portion and…

This report identifies the methods and issues in the evaluation of student writing in classrooms and large scale assessments using an analytical scoring system from the Illinois Inventory of Educational Progress (IIEP). This guide is written for staff in state offices of education conducting large-scale writing assessments, and also for personnel…

The benefits of using essay tests rather than objective tests in professional education programs are discussed. Essay tests offer practice in writing, creativity and formal communications. Guidelines for using and scoring a sample essay test in biology are presented. (BH)

A follow-up study of two instruments for evaluating college writing was conducted. The experimental scale (E Scale) was developed in 1976 and revised for this study. The control scale (C Scale) was described in the literature in 1977. Ten English majors graded ten essays from diagnostic entrance exams. Both the E Scale and the C Scale were used,…

The effectiveness of the "read-comp" technique in assessing writing ability and the usefulness of a rubric and procedure devised for scoring read-comp samples and essays were evaluated. Subjects were 100 freshman students enrolled in general and remedial English classes in a 6-week summer session at Indiana University of Pennsylvania.…

The purpose of this study was to develop a testing method for the assessment of various types of writing at the elementary school level that would meet acceptable standards for educational measurement instruments as well as standards of utility and feasibility within a given educational system. The study was conducted within the framework of an…