NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED306232
Record Type: Non-Journal
Publication Date: 1989-Mar
Pages: 22
Abstractor: N/A
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Variation among Examiners and Protocols on Oral Examinations.
Lunz, Mary E.; And Others
A method for understanding and controlling the multiple facets of an oral examination (OE) or other judge-intermediated examination is presented and illustrated. This study focused on determining the extent to which the facets model (FM) analysis constructs meaningful variables for each facet of an OE involving protocols, examiners, and candidates; and establishing that the OE data fit the FM. The FM for calibrating all facets of an examination is an expansion of the Rasch model. The model was applied to data collected from an OE administered by a medical specialty board to 293 candidates. The same 48 examiners judged candidates on 18 protocols of case management, examining each candidate on three protocols. The FACETS computer program for the Rasch analysis of examinations with multiple facets was used to analyze the data. The FACETS program produces three statistics for each element of each facet: (1) a logit measure; (2) its standard error; and (3) the model fit statistic (infit and outfit). Results show that variables for protocol difficulty and examiner severity can be constructed. Examiners varied markedly in their level of severity, but were consistent in that level across candidates. Protocols and examiners generally fit the expectations of the model. When a protocol or examiner does not perform as expected, the fit statistic flags the situation. The FM applies a complex and powerful analysis to the examination situation that requires a series of human judgments combined with item writing methodology, rating scale definitions, and candidate interviews. Three data tables and one graph illustrate application of the model. (SLD)
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: American Society of Clinical Pathologists, Chicago, IL.
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A