ERIC Number: ED494160
Record Type: Non-Journal
Publication Date: 2006-Feb-25
Pages: 22
Abstractor: Author
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
How Good Are Our Raters? Rater Errors in Clinical Skills Assessment
Iramaneerat, Cherdsak; Yudkowsky, Rachel
Online Submission, Paper presented at the Graduate Educational Conference, Education and the Public Good: Interdisciplinary Trends in Graduate Scholarship (Chicago, IL, Feb 2006) February 2006
A multi-faceted Rasch measurement (MFRM) model was used to analyze a clinical skills assessment of 173 fourth-year medical students in a Midwestern medical school to investigate four types of rater errors: leniency, inconsistency, halo, and restriction of range. Each student performed six clinical tasks with six standardized patients (SPs), who were selected from a pool of 17 SPs. SPs provided ratings of the performance of each student in six skills: history taking, physical examination, interpersonal skills, communication technique, counseling skills, and physical examination manner. Participating SPs showed statistically significant differences in their rating severity, indicating rater leniency error. Four SPs exhibited rating inconsistency, as evidenced from their high infit mean-square statistics. None of the SPs had low infit mean-square statistics, indicating that they did a good job in avoiding halo and restriction of range errors. These findings provided diagnostic information which was useful for a medical school to guide the ways to improve the quality of subsequent clinical skills assessments by eliminating construct-irrelevant variance caused by rater errors. (Contains 2 tables and 1 figure.)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A