Problem Statement: In addition to being teaching tools, concept maps can be used as effective assessment tools. The use of concept maps for assessment has raised the issue of scoring them. Concept maps generated and used in different ways can be scored via various methods. Holistic and relational scoring methods are two of them. Purpose of the…

Automated scoring models for the "e-rater"® scoring engine were built and evaluated for the "GRE"® argument and issue-writing tasks. Prompt-specific, generic, and generic with prompt-specific intercept scoring models were built and evaluation statistics such as weighted kappas, Pearson correlations, standardized difference in…

This study examined the effects of marking method and rater experience on ESL (English as a Second Language) essay test scores and rater performance. Each of 31 novice and 29 experienced raters rated a sample of ESL essays both holistically and analytically. Essay scores were analysed using a multi-faceted Rasch model to compare test-takers'…

Reports possible improvements in debate ballots beyond the American Forensic Association "Form C." (CH)

Investigated psychometric properties of three methods of scoring a Mixed Standard Scale performance evaluation: a patterned procedure, simple nonpatterned scoring procedure and procedure assigning differential weights to statements on the basis of scale values provided by subject matter experts. Found no differences in the score distribution…

This report presents the results of a research and development effort for SpeechRater? Version 1.0 (v1.0), an automated scoring system for the spontaneous speech of English language learners used operationally in the Test of English as a Foreign Language™ (TOEFL®) Practice Online assessment (TPO). The report includes a summary of the validity…

The letter unit and the whole word method were used to score the spelling of 70 third graders (35 good spellers and 35 poor spellers). Results indicated that either scoring procedure could have been used. Use is suggested based on needs for word analysis or for quick scoring time. (CL)

The essay on the writing section of the SAT will be scored using a holistic approach. In holistic scoring, a piece of writing is considered as a total piece of work--the whole of which is greater than the sum of its parts.

This study identifies critical elements of the front dive half-twist that judges are likely to look for when they score springboard diving competitions. Videotapes of divers at a 1979 intercollegiate diving meet were made and analyzed, using grid scoring procedures, to isolate components of the dives that would help predict judges' scores.…

This study evaluated a "substantively driven" method for scoring NAEP writing assessments automatically. The study used variations of an existing commercial program, e-rater[R], to compare the performance of three approaches to automated essay scoring: a "brute-empirical" approach in which variables are selected and weighted solely according to…

This report identifies the methods and issues in the evaluation of student writing in classrooms and large scale assessments using an analytical scoring system from the Illinois Inventory of Educational Progress (IIEP). This guide is written for staff in state offices of education conducting large-scale writing assessments, and also for personnel…

Standardized testing plays an increasingly important role in the lives of today's students and educators. The U.S. No Child Left Behind Act (NCLB) requires assessment in math and literacy in grades 3-8 and 10 and, as of 2007-08, in science once in grades 3-5, 6-9, and 10-12. Based on National Center for Education Statistics enrollment projections,…

The second section of a four-part technical report of Florida's statewide program for assessing reading-related skills in grades 2 and 4 provides statistical and scoring information. Item and student sampling, a test re-scoring study, reporting statewide assessment data (percentage of achievement, types of tables used), and standard error…

Four systems for scoring student essays were used to classify eleventh grade and undergraduate students according to writing ability. The reliabilities of the raters and the validities of the systems in classifying students were emphasized. Two analytic scoring systems--which assume that quality writing is characterized by the inclusion of certain…

This paper outlines the development of techniques for computer-based personality assessment from sentence completions. The One-Word Sentence Completion (OWSC) instrument was designed to elicit data suitable for machine processing, while retaining most of the advantages of a free-response format. Two operative scoring systems are described. The…