Three studies are reported addressing the reliability of the Scale for Assessing Emotional Disturbance (SAED), a standardized, norm-reference measure linked to the federal definition of emotional disturbance (ED). Results indicate the SAED possesses acceptable test-retest reliability and reasonable interrater reliability and can assist in the…

The consistency between raters over three years of a high-stakes performance assessment was examined in two studies involving a total of approximately 3,000 students in grades three, five, and eight. Results show that raters in different years differ in severity, with raters in mathematics most consistent, and those in language arts least…

The development of the Asperger Syndrome (and high-functioning autism) Diagnostic Interview (ASDI) is described. Preliminary data from a clinical study of 20 individuals (ages 6-55) suggest that interrater reliability and test-retest stability may be excellent, with kappas exceeding 0.90 in both instances. The validity appears to be relatively…

Multiple observer ratings of 424 families were obtained across 2 observational task situations using the Iowa Family Interaction Rating Scales. Observer ratings and family member reports were assessed simultaneously through structural equation modeling. Findings support the reliability of the global assessments of warm/supportive marital…

Reliability analysis of the Motivation Assessment Scale was conducted on subscales completed by staff members working with 47 institutionalized adults with severe to profound mental retardation and self-injurious behavior problems. Internal consistency was found to be superior to interrater reliability. The instrument's internal consistency…

Studied the stability of rater severity over an extended rating period by applying multifaceted Rasch analysis to ratings of 16 raters of writing performances of 8,285 elementary school students. Findings cast doubt on the practice of using a single calibration of rate severity as the basis for adjustment of person measures. (SLD)

Compared reliability of three widely used child protective service risk-assessment models (one actuarial, two consensus based). Found that, although no system approached 100% interrater reliability, raters employing the actuarial model made consistent estimates of risk for a high percentage of cases they assessed. Interrater reliability for the…

In study 1, 313 supervisor/supervisee pairs rated supervisees' professional expertise; supervisees gave themselves higher ratings. Study 2 compared 63 team leaders' and 593 team members' ratings of leaders, finding different perceptions of competence. Results suggest the use of self-other ratings can be improved through feedback, joint training…

This paper reports on findings relating to a project on gender and essay assessment in HE. It focuses on one aspect of the study: the assessment of and feedback given to two sample essays by 50 historians based at universities in England and Wales. We found considerable variation both as to the classification awarded to the essays and to positive…

The authors describe psychometric characteristics of the new Therapy Process Observational Coding System-Alliance scale (TPOCS-A; B. D. McLeod, 2001) and illustrate its use in the study of treatment as usual. The TPOCS-A uses session observation to assess child-therapist and parent-therapist alliance. Both child and parent forms showed acceptable…

Although effective, humane treatments exist for persons with intellectual disabilities (ID) who have challenging behaviour, little research has examined the extent to which clients receive formal, documented vs. undocumented interventions. Caregivers (of 625 persons with ID living in community and institutional residences in Ontario, Canada) were…

In this article, we extend previous work with respect to interrater reliability measure of computer-mediated conferencing and suggest coding categories relevant to problem-based learning. Calculating interrater reliability agreement by using a Transcript Reliability Cleaning Percentage (TRCP) approach is simple for academics with limited…

Thirty-four undergraduates used Web-based self- and peer-assessment procedures for evaluating proposals in experimental psychology courses. Students presented their proposals and commented on the proposals of others on the Web. Results indicated that proposal observation and peer interaction enhanced the quality of students' proposals. These…

Studies of self-induced vomiting of retarded persons have found that the rate of eating and the amount eaten alter this problem. The present study attempted to determine whether this same relationship was exhibited by the nonretarded bulimic. A nonretarded bulimic woman provided her subjective ratings of her desire to vomit after eating her taboo…

This study reports on norms developed for the Minnesota Institute of Public Health's (1999) Community Readiness Survey. Prevention experts from ten states and the Red Lake Nation sorted data from 50 communities into high and low readiness groups using a Q-sort process. High inter-rater agreement was achieved on communities sorted. Tests of…