The method of paired comparisons was used to rank 42 dental students on their performance in emergency and screening clinic rotations. Results suggest this methodology may provide more internally consistent student assessments on more subtle aspects of clinical performance than those assessed by multiple-choice tests or written performance…

Examined factors that influenced the concordant perceptions of 28 distressed and 28 nondistressed husbands and wives and trained coders who observed samples of their own and another couple's problem solving. Correlational analyses suggested greater insider-outsider perceptual agreement for distressed than nondistressed couples and for negative…

Teachers and classroom aides of learning disabled pupils were asked to complete the Conners Abbreviated Teacher Rating Scale (CATRS) on two separate occasions, one month apart. Inter-rater reliability for teachers (.866) and for aides (.602), and reliability across time for teachers (.866) and aides (.603) achieved acceptable levels. (Author/BW)

This paper describes a study on rater training that involved the analysis of ratings given to English-as-a-Second-Language (ESL) compositions by 8 inexperienced and 8 experienced raters both before and after rater training, using FACETS (Linacre, 1990, 1993), which provides measures of rater severity and consistency. The testing text was a…

Studied interviewer reliability, validity, and strategy for information integration. Candidates (N=412) for selection to a military division were interviewed and assessed. Results indicated that interviewers functioned in a similar fashion. Analyses of individual interviewers indicated higher reliability and individual differences among…

Results and interpretations of the data from a performance examination were compared for four methods of analysis for 74 medical specialty certification candidates: (1) traditional summary statistics; (2) inter-judge correlations; (3) generalizability theory; and (4) the multifaceted Rasch model. Advantages of the Rasch model are outlined. (SLD)

Reports on the interrater reliability of a language arts portfolio assessment in the primary grades of the Rochester (New York) school system. Results from approximately 400 primary grade portfolios rated by 2 raters show that teachers can assess their own students' work reliably. (SLD)

The Observation Procedure for Technology Education Mental Processes, a computerized assessment tool, was based on duration and frequency of mental processes needed for problem solving. Videotapes of students completing problem-solving activities were used to identify the processes. Interrater reliability tests validated the program. (SK)

The influence of information level on the production of accuracy of 20 children was examined. Data were children's productions of nouns in sets of utterances referring to triplets of pictures representing noun-verb-noun utterances. (Author/JL)

Grading profiles for formative and summative assessment in a British nursing school were designed with criterion referencing to improve validity and interrater and intercourse reliability. Assessment was conceptualized as an ethical activity that clarifies expectations through specification of criteria. (SK)

In a study of criteria for and reliability of grading of college essays in introductory psychology, 16 essays were marked by 12 faculty and 20 graduate students. Analysis found that two content attributes (facts, examples) accounted for 82% of variance in grading by faculty, while five stylistic measures accounted for the remainder. Both faculty…

A study involving 30 Argentineans with autism evaluated the validity of the Autism Diagnostic Inventory-Telephone Screening in Spanish (ADI-TSS). The final version of the ADI-TSS could be assessed in 20 to 40 minutes and demonstrated a high validity, high interrater reliability, and high internal consistency. (Contains references.) (Author/CR)

Considers the development of an inter-rater reliability correlation comparing the judgments, or scores, or each judge to see if their observations are similar. Presents a case study of the Northville Public Schools' data for the 2000 MEAP (Michigan Educational Assessment Program) Writing Test. Concludes that in this case study the state fails both…

Examines the question of variation among interviewers of oral language proficiency interviews in the ways that they elicit demonstrations of communicative ability and the impact of this variation on candidate performance and raters' perceptions of candidate ability. A discourse analysis of two interviews involving the same candidate with two…

Examinations were analyzed using the Rasch model to determine differences in judge severity and grading period stringency for (1) essay examination (subjects were 12 judges and 32 examinees); (2) clinical examination (subjects were 18 judges and 217 examinees); and (3) oral examination (subjects were 46 judges and 270 examinees). (SLD)