The use of online rater self-training is growing in popularity and has obvious practical benefits, facilitating access to training materials and rating samples and allowing raters to reorient themselves to the rating scale and self monitor their behaviour at their own convenience. However there has thus far been little research into rater…

Behaviorally Anchored Rating Scales (BARS) were developed to score responses from a previously designed police written communication test that lacked reliability. Rating scales for each of the 9 dimensions of the test consisted of the scale definition and a 5-point continuum, with the scores of 5, 3, and 1 defined by specified behavioral…

Philosophy statements have been used in the National Debate Tournament (NDT) since the mid-1970s and the Cross Examination Debate Association (CEDA) National Tournament since its 1986 inception. The statements should help debaters adapt to critics' expressed preferences. Moreover, philosophy statements can guide the study of argumentation theory…

The objectives of this study were to determine whether student writing portfolios could be rated reliably by trained judges; study the effects on student ratings of the differential leniency of the judges; and ascertain the effects of writing-prompt difficulty and its interactions with rater leniency. Writing samples from 127 students in grades 3,…

A study addressed issues of concern in the use of the American Council on the Teaching of Foreign Languages (ACTFL)/Educational Testing Service (ETS) Language Proficiency Guidelines commonly used in determination of oral language proficiency. Specifically, potential discrepancies between the judgments of trained raters and "naive" native…

A protocol for analyzing the activity structure of a classroom lesson was developed and field tested. The protocol can be completed on site by a person serving as both observer and analyst. Validity of the protocol was established by reference to the large body of qualitative research on activity structure and the analysis of teaching. The…

The study compared maternal and trained observer evaluations of infant temperamental characteristics, to determine how closely the ratings correspond, and to analyze the impact of maternal drug abuse habits on maternal ratings of infant temperament. In relating observer to maternal ratings of infant temperament, seven dimensions were compared:…

Two factors which may affect the ratings assigned to an essay test are investigated: (1) context effects; and (2) score level effects. Context effects exist in essay scoring if an essay is rated higher when preceded by poor quality essays than when preceded by high quality essays. A score level effect is defined as a change in the score (value)…

Rating scales are a typical method for evaluating a student's performance in outcomes assessment. The analysis of the quality of information from rating scales poses special measurement problems when researchers work with faculty in their development. Generalizability measurement theory offers a set of techniques for estimating errors or…

The accuracy of standards obtained from judgmental methods is dependent on the quality of the judgments made by experts throughout the standard setting process. One important dimension of the quality of these judgments is the consistency of the judges' perceptions with item performance of minimally competent candidates. Several interrelated…

A total of 1,487 eleventh grade students from the Hamburg (West Germany) school system were asked to complete four writing assignments used in an International Association for the Evaluation of Educational Achievement (IEA) study of writing assessment. In analyzing the writing samples, the study focused on: (1) between-rater effects; (2)…

A taxonomy for literature reviews in education and psychology is presented. The increased use of the descriptor "literature review" in ERIC and Psychological Abstracts documents between 1969 and 1983 is cited as creating the need for categorization. The taxonomy categorizes reviews according to focus, goal, perspective, coverage,…

A scale was developed and tested to measure metaphoric complexity in order to aid teachers in the selection and sequencing of young adult (YA) novels in the English curriculum. The development and testing of the scale involved two stages: (1) the development of a questionnaire to determine if differences in the metaphoric complexity of examples…

The terms reading disability and dyslexia are discussed, as well as the meaning of function analysis as a way of diagnosing behavior and difficulties in reading and spelling. The author's model classifies reading disability and dyslexia as auditory, auditory-visual, visual, emotional, or pedagogic. The Bergen Study is described. (Author/LMO)

Questions the extent to which assessment of solo musical performance can be made under the General Certificate of School Education exam in England and Wales. Discusses performances as criterion. Reports on experiment which attempted to assess a student's overall music performance. Offers a model which can be used to better measure solo music…