Developed method for aggregating psychodynamic formulations of independent clinicians. Panels of clinicians observed videotaped interviews of patients and wrote individual formulations which were combined into consensual formulation. Other clinical raters read each consensual formulation and judged whether each problem was apt to be distressing…

To test model of general managerial effectiveness, superiors (N=271), subordinates (N=605), and peers (N=469) rated 344 managers. Study designed to test three specific hypotheses on criterion type and criterion weights found consensus in effectiveness models of superiors, subordinates, and peers. Consensus among different raters was high on both…

New coefficients of agreement are suggested for the measure of intraclass consistency between observations on two variables. The coefficients are derived from a general coefficient for measuring intraclass dependence in a bivariate analysis context. Various coefficients for the univariate agreement analysis are shown to be cases of the suggested…

Examined interrater reliability of diagnoses made on basis of structured interview for psychiatric patients with and without psychoactive substance use disorders (PSUDs). Results from 47 pairs of ratings by 9 clinical interviewers revealed that interrater reliability for non-PSUD psychiatric diagnoses was quite high when patient had no diagnosable…

A coding method for measuring reflective thinking in student journals was tested twice, demonstrating acceptable reliability among evaluators and supporting the precision of the guidelines for coding. Coding categories were as follows: habitual action, introspection, thoughtful action, content reflection, process reflection, content and process…

To examine the interrater reliability of the Ruff Figural Fluency Test (RFFT) (R. Ruff, 1988), 124 college students completed the measure and scored RFFT test protocols. Results indicated substantial interscorer reliability on the RFFT, particularly for number of unique designs. Reliability was lower for scoring perseverative errors and error…

Three commercial job evaluation firms rated the same set of 27 jobs. Statistical analysis indicated that evaluators differed in which job traits they used to evaluate inherent job worth. Comparable worth may thus be sensitive to the choice of evaluator. (Contains 24 references.) (Author/SK)

Extends the reliability research of the Qualitative Scoring System for the Modified Version of the Bender-Gestalt Test. The test was administered to 48 children, and the 48 test protocols were scored independently by two psychologists using the Qualitative Scoring System. Results indicated that the scoring system is highly reliable. (JPS)

Registered nurses (n=368) participated in a skills recredentialing program in which competencies were assessed by a knowledge test and performance test under simulated conditions and evaluator ratings in actual patient-care situations. No significant differences in results between the simulated and actual conditions support the validity of the…

Assessed the interrater reliability of the revised Advanced Psychodiagnostic Interpretation (API) (A. Raphael and C. Golden, 1998) scoring system for the Bender Gestalt Test (L. Bender, 1938). Agreement across nine raters exceeded 90% for each of three clinical protocols, and kappa statistics indicated good interrater reliability. (SLD)

Reliability and validity of scores from management seminars using a multirater feedback system were tested. Ratings of participants affected interrater reliability negatively, suggesting that formative and summative stages be held separately. (SK)

Three studies assessed the accuracy of individuals' self-estimates of competence, using a concept of relative competence to account for rater variations. Results indicate that self-estimates can be accurate and the individual's perception of how important specific competencies are to the performance of certain jobs should be taken into account.…

Objective: Autism is a diagnostic spectrum of variable severity, with significant comorbidity. No existing standardized interview measures autistic features dimensionally. The authors aimed to develop a parental autism interview that could be administered to unselected clinical and general population samples that measures both symptom intensity…

Objective: To present data from a general infant psychiatry clinic, including range and frequency of presenting symptoms, relationship between symptoms and diagnoses, and comparison of two diagnostic systems, DSM-IV and Diagnostic Classification of Mental Health and Developmental Disorders of Infancy and Early Childhood (DC: 0-3). Method: A…

Objective: The purpose of this study was to determine the level of inter-rater reliability of the Illinois Structured Decision Support Protocol by examining the level of Child Protective Services (CPS) caseworkers' agreement regarding state interventions. The Protocol was designed to guide CPS workers to consistent decisions related to the level…