A study of 71 students (ages 7-17) with disabilities investigated the interrater agreement of the Adjustment Scales for Children and Adolescents (ASCA), a behavior rating scale used in school settings. Participants were rated by 29 educational professionals in 24 classrooms. Results found ASCA produced acceptable levels of interrater agreement.…

Presents procedures based on Rasch measurement theory for construction of an assessment network, which is defined as a system of rate and task banks. Three general classes of data collection designs are presented to calibrate an assessment network and provide the opportunity for objective and fair measurements. (SLD)

Examines artwork of elementary school students to determine the interrater and inter-item reliabilities of ratings given by professional artists. The generality of creative performance for the artworks, and the topic of age trends in artistic creativity, are also considered. (BB)

Twenty rehabilitation patients were assessed on their activities of daily living using the Barthel Index, and were also observed by two occupational therapists in a simulated home unit. Results indicated good inter-observer reliability, and good agreement between asking the patient and professional observation of the patient. (JDD)

A method of assessing rater reliability via a design of overlapping rater teams is presented. Covariances or correlations of ratings can be analyzed with LISREL models. Models in which the rater reliabilities are congeneric, tau-equivalent, or parallel can be tested. Two examples based on essay ratings are presented. (TJH)

The effects on essay scores of intermingling handwritten and word-processed student essays were studied with 32 students who produced handwritten and word-processed essays. Essays were converted to the other format and rescored. Results reveal higher average scores for handwritten essays. Implications for scoring are considered. (SLD)

Two longitudinal studies involving 29 raters concerning the construct validity, temporal stability, and interrater reliability of the latent common factors underlying subjective assessments by human raters of personality traits in the stumptail macaque and the zebra finch illustrate the use of generalizability analysis to test prespecified…

The degree of agreement between two raters rating several objects for a single characteristic can be expressed through an association coefficient, such as the Pearson product-moment correlation. How to select an appropriate association coefficient, and the desirable properties and uses of a class of such coefficients--the Euclidean…

Sixty-five pairs of teachers and parents of elementary and secondary school learning-disabled students completed the Homework Problem Checklist (HPC). The HPC demonstrated a moderate level of interrater reliability. Acceptable levels of internal consistency were reported for both teacher and parent ratings. (JDD)

Tested the assumption that familial self-reports are biased by social desirability and other factors, through the use of a latent variables modeling approach that evaluated rater reliability and bias in mother, father, and child ratings of parent-child negativity. Results based on 78 families demonstrated that family member ratings contained a…

National Assessment of Educational Progress writing sample essays from 1988 and 1990 (495 and 599 essays) were subjected to computerized grading and human ratings. Cross-validation suggests that computer scoring is superior to a two-judge panel, a finding encouraging for large programs of essay evaluation. (SLD)

A peer evaluation experiment in a British polytechnic institute found peer grading correlated positively with teacher grading. A survey of participants (n=70) found that five advantages of peer marking had been achieved and that students believed their work had been marked fairly and the marks should count toward final grades. (Author/MSE)

A preprinted evaluation form and generalizability theory were used to judge the reliability of student grading of their peers' papers. Findings suggest that students can be consistent and fair in their assessments. Student practice in peer evaluation will help develop the management skill of employee evaluation. (JOW)

The use of paraprofessional editors to evaluate student writing, particularly in large college classes, allows teachers to give students more writing practice, provides more individual assistance for students, and helps teachers gain insight into student needs. Adoption of uniform criteria for evaluation also provides consistency and objectivity.…

Interrater reliability in numerical ratings of clerkship performance (n=1,482 students) in five surgery programs was studied. Raters were classified as accurate or moderately or significantly stringent or lenient. Results indicate that increasing the proportion of accurate raters would substantially improve the precision of class rankings. (MSE)