Reports on a study of 614 experienced music teachers, non-music teachers, college-level music students, and non-music students on the effect of sequential patterns and different modes of presentation of music teaching. Finds that experienced teachers' evaluations were significantly higher than those of university students. (CFR)

A study found that interviewers of medical school applicants (n=471) were influenced in their ratings of applicants' noncognitive attributes by grade point average and Medical College Admission Test scores, when available, and by gender and race in accordance with affirmative action goals. Only moderate reliability across interviewers was found.…

A study compared student and trained observer ratings of 15 high-rated and 15 low-rated lecturers in a multi-instructor medical course to identify distinguishing delivery characteristics. Student ratings were stable over three years; trained observers discriminated between students' highest- and lowest-rated lecturers. Voice presentation was the…

This study investigated two pragmatic profiles (the Pragmatic Profile and the Profile of Communicative Appropriateness) used to assess the language of two aphasic patients. The study examined interscorer reliability, scoring sensitivity, and diagnostic accuracy. Findings indicate that training in scoring these profiles must be uniform, and greater…

Two studies investigated interrater agreement among 10 clinical dental examiners who scored residual subgingival calculus after student scaling on 4,160 real and 92 manikin tooth surfaces. Interrater reliability was low. Results suggest a need in periodontics for effective examiner calibration methods and objective subgingival calculus detection…

Possible sources of intrajudge inconsistency in standard setting are reviewed, and approaches are presented to improve the accuracy of rating. Procedures for providing judges with feedback through discussion or computerized communication are discussed. Monitoring and maintaining judges' consistency throughout the rating process are essential. (SLD)

Two frame-of-reference rater training approaches were compared for effects on reliability and accuracy of cutoff scores generated by 21 raters using Angoff methods on tests taken by 155 undergraduates. Both approaches result in higher interrater reliability and more accuracy than does a non-frame-of-reference method. (SLD)

To assess cross-observer agreement on personality profiles, an Index of Profile Agreement and an associated coefficient are proposed that take into account both the difference between the ratings and the extremes of their mean. Data from the Revised NEO Personality Inventory for 250 peer ratings/self-reports and 68 spouse ratings/self-reports…

A meta-analysis of studies examining the interrater reliability of the standard practice of peer assessments of quality of care was conducted through the use of several databases. The mean weighted kappa of 21 findings from 13 studies was 0.31, which suggests that the interrater reliability of peer assessment is limited. (SLD)

Discusses the rationale for, and end results of, a small-scale study of the effects of learners' creativity on language learning. The study was part of an ongoing, larger-scale project on individual differences at Eotvos University in Budapest, Hungary. (Author/VWL)

This article cautions against complacency in "subjective" assessment, arguing that even tests designed to reflect the development of learner-centered, interactive and communicative approaches to teaching English may have cultural bias built into their assessment criteria. The reply article singles out as an unresolved issue whether or…

A study compared evaluations of undergraduate students' oral presentations by groups of peers and faculty teams. Students (n=210), in assessment teams, rated quality of content and presentation using a 22-point guide. Comparison of peer and faculty scores showed no significant difference in averages but did show significant differences in standard…

Investigated the reliability and item appropriateness, as discerned by adults affiliated with an infant center, of the Caregiver-Environment Scale (CES). Found the CES to be an easy to use, reliable instrument for evaluation. (Author/SD)

Describes how some background factors influenced the way in which one experienced rater dealt with a number of operations involved in setting up and applying scoring criteria in the assessment of 60 Test of English as a Foreign Language essays. Implications are drawn for both future research into interrater variability and for rater training.…

This paper reports the results of a special urban study of the 2005 National Assessment of Education Progress which indicates that city school districts may be seeing some payoff from years of work to improve mathematics instruction. However, similar initiatives to raise reading achievement have not led to significant gains. While, most of the 11…