An asymmetric version of J. Cohen's kappa statistic is presented as an appropriate measure for the agreement between two observers classifying items into nominal categories, when one observer represents the "standard." A numerical example with three categories is provided. (SLD)

This study examined interjudge agreement levels for five adult listeners assessing either overt stuttering or disfluency types in the spontaneous speech of eight young children. Results showed that the interjudge reliability for judgments based on a disfluency taxonomy was not significantly different from that based on stuttering. The importance…

Developed a video portfolio technique of teacher assessment and evaluated the technique through studies of six teachers and their raters. Results show that teachers are consistent in observing teaching functions and using their observations to evaluate teaching. (SLD)

Behavior measures and the Myers Briggs Type Indicator were completed by 343 senior managers; 3,158 of their peers, supervisees, and supervisors rated managers' behavior. A modest correlation appeared between personality type and manager behavior. Differences related to raters' perceptions were found. (SK)

This study examined the inter-rater reliability of clinician-assigned diagnosis of autism using or not using the criteria specified in the Diagnostic and Statistical Manual IV (DSM-IV). For experienced raters there was little difference in reliability in the two conditions. However, a clinically significant improvement in diagnostic reliability…

Suggests that a modification of traditional linking is necessary when tests consist of constructed response items judged by raters and a possibility of year-to-year variation in rating discrimination and severity exists. Illustrates this situation with an artificial example. (SLD)

Self, subordinate, peer, and supervisor ratings of 1,080 managers in education, military, government, manufacturing, finance, and health were analyzed for leniency, interrater agreement, and effectiveness. In the private sector, more poor performing managers tended to overestimate their performance. Interrater agreement was lowest in government…

Evaluates the effects of rater reliability of common IQ and achievement tests on subsequent learning disorder eligibility determinations, particularly with respect to difficulty level of individual subtests and expertise of the scorer. The study corroborates previous findings of strong interrater reliability on most subtests of common IQ and…

In this study, 30 judges identified disfluency types they perceived in audiovisually recorded speech stimuli, first individually and then with a partner. Although intrapair and interpair agreement was higher in the partner than the individual condition, agreement for occurrences still averaged below 50 percent. Findings suggest caution in use of…

A coefficient of unfairness in the allocation of goods to people can be extended to measuring consensus among judges. The notion of relative variability underlies the formation of these measures.

This article discusses the rationale for, and explicates the process used in, developing differentiated authentic assessments for middle school classrooms (many of which contain gifted students) that are aligned with state academic standards. The assessments were developed based on learner-centered psychological principles and revised based on a…

This article addresses the problem of accounting overall multivariate chance-corrected interobserver agreement when targets have been rated by different sets of judges (not necessarily equal in number). The proposed approach builds on Janson and Olsson's multivariate generalization of Cohen's kappa but incorporates weighting for number of judges…

This paper comments on the response offered by Szechtman and Woody to Taylor et al's initial comments on Szechtman and Woody's original article. Taylor et al highlight one problem with their model that Woody and Szechtman seem to think is unimportant: the treatment relevance of their model. The analogy of aspirin and colds was used, suggesting…

This review examines interventions intended to improve the literacy functioning of adolescent students with emotional and/or behavior disorders. Seventeen studies met inclusion criteria and included a variety of interventions designed to affect a variety of literacy areas, including spelling, writing, and reading fluency. Findings from these…

Until recently, research studies have implied that domestic violence does not affect Asian American and immigrant communities, or even Asians abroad, because ethnicity or culture has not been addressed. In this content analysis, the authors examined trends in publications in leading scholarly journals on violence relating to Asian women and…