To determine if there is a distinction between the forensics community's idea of quality and that of the general population, tournament rankings of forensics judges and those of a lay audience were compared. Undergraduate students enrolled in a variety of speech related courses were asked to attend rounds of competition at a midwest collegiate…

This paper describes the Measure of Epistemological Reflection (MER), an instrument to assess cognitive developmental level according to the Perry scheme of intellectual and ethical development. It contains sets of questions for each of the six cognitive domains: decision making, learner role, instructor role in the learning process, peer role in…

The occurrence of discrepant, or non-contiguous, scores during holistic scoring of writing proficiency is usually blamed on variables of the test reader or external factors. It is hypothesized that characteristics of the essays themselves sometimes may generate discrepancies in essay scoring. Data from 152 essays of college sophomores in teacher…

The question of whether weighting (rank-ordering) of student teacher evaluation criteria, as determined by student and supervising teachers, university supervisors, and employing officials, results in better differentiation between and among student teachers is explored in this paper. Participants in the study were asked to rank order 15…

The hypothesis that first-year medical residents' evaluate their performance in a more differentiated manner than their supervisors was evaluated. An evaluation form was developed to obtain ratings of performance in 13 specific areas. Results showed that residents were more discriminating than supervisors in evaluating residents performance. (JAZ)

This study examined halo in raters' beliefs of item (attribute) relatedness. College students' prior beliefs of the co-occurrence of teaching attributes were correlated with actual correlation of teaching attributes of fictional college professors. Results showed some support for beliefs-of-relatedness source of halo. (LMO)

Demonstrates the inadequacy of students' rating of classroom speeches: student raters were both unreliable and sources of considerable error. Emphasizes the importance of training to reduce error rate. (PD)

This publication presents the reports from two studies, Math Online (MOL) and Writing Online (WOL), part of the National Assessment of Educational Progress (NAEP) Technology-Based Assessment (TBA) project. Funded by the National Center for Education Statistics (NCES), the Technology-Based Assessment project is intended to explore the use of new…

This report describes ongoing research investigating the use of teachers' assignments as an indicator of classroom practice. The purpose of this work is to develop a measure of students' learning environments that potentially could be used to help monitor the influence of school reform efforts on the quality of the classroom learning environment…

As schools move toward performance assessment, there is increasing discussion of using these assessments for accountability purposes. When used for making decisions, performance assessments must meet high standards of validity and reliability. One major source of unreliability in performance assessments is interrater disagreement. In this paper,…

Elementary school pupils judged their student teachers' performances in the areas of planning, instructional skill, evaluation, and behavior, and in classroom organization and control. Their evaluations were compared with adult observers' ratings. Results indicate that elementary school students can assess performance and discriminate among…

This study examined how local content specialists performed when applying the Angoff and Nedelsky standard setting procedures to objective-referenced instruments in reading and mathematics. Results revealed several differences between the standard setting procedures in terms of both level and consistency of the cut scores generated. (Author/BW)

Continuous observational recording over 57 hours evaluated behavior states of three adults with profound and multiple disabilities. Two independent observers also recorded for 22 hours. Although overall percentage agreement was satisfactory (above 80%), agreement on occurrence was unsatisfactory (mean of 65%). Agreement data were superimposed on…

Development of the Stereotyped Behavior Scale for adolescents and adults with mental retardation is described. Use with 600 individuals resulted in refinement and a 26-item scale with an internal consistency alpha of 0.88, test-retest reliability of p=0.90, and interrater reliability of p=0.76. (DB)

Discusses concerns and research about college student peer evaluation, particularly with regard to gender bias. Reports a study using blind marking, and examines results in relation to task and other contextual variables. Concludes that the blind marking technique contributes to the reliability of peer assessment, and outlines additional…