To test the effect of clinically relevant multiple-choice item content on the validity of statistical discriminations of physicians' clinical competence, data were collected from a field test of the Emergency Medicine Examination, test items for the certification of specialists in emergency medicine. Two 91-item multiple-choice subscales were identified: medium difficulty (MD)--items of ideal difficulty for norm-referenced achievement tests, and high clinical-relevance (HCR)--items that correlated highly with an independent criterion measure of simulated clinical performance. A stepwise discriminant analysis of these two subscale scores showed that HCR was more statistically discriminating of residency-eligible physicians, second-year residents, and fourth-year medical students than was the MD scale. The HCR scale was significantly more reliable and significantly less difficult than the MD scale. It was concluded that: (1) multiple-choice examinations, calibrated to a simulated clinical performance measure, can be very valid discriminators of levels of clinical competence; and (2) selection of multiple-choice items for medium difficulty is less efficient than selection of items for their high correlation to a simulated clinical performance measure. (Author/CP)