The study examines the agreement among raters on children's problematic behaviors. A multitrait-multimethod (MTMM) matrix was applied to a normative sample of elementary school-aged children (N = 841). The participants were rated by their physical educators, using the Motor Behavior Checklist for children (MBC; Efstratopoulou, Janssen, &…

Some suggestions for measuring marginal symmetry in agreement matrices for categorical data are discussed, together with measures of item-by-item agreement conditional on marginal asymmetry. Connections with intraclass correlations for dichotomous data are noted. (Author)

When constructed response test items are scored by more than one rater, the repeated ratings allow for the consideration of individual rater bias and variability in estimating student proficiency. Several hierarchical models based on item response theory have been introduced to model such effects. In this article, the authors demonstrate how these…

The degrees of variance among three groups of evaluators relative to their assessments of the teaching competencies of preservice teacher education students were studied. Subjects included groups of 23 and 32 undergraduates who were certified to teach by the teacher preparation program at Washburn University in Topeka (Kansas) in 1987 and in 1988,…