A study was carried out to ascertain whether language instructors or naive judges were more reliable in judging oral proficiency. Fifteen students were chosen from the ESL (English as a second language) center at Southern Illinois University to record a tape while reading passages in English. The tape and a questionnaire were administered to 70 raters. Half of them were students who had neither linguistic nor teaching experience (the naive group) while the other half were instructors or teaching assistants in ESL (the experienced group). Each sample was rated by the judges on each of six scales: the first four scales consisted of four pairs of bipolar adjectival descriptors, an overall proficiency scale, and a multiple choice questionnaire about the language background of the speaker. There was a very substantial agreement among the raters, regardless of whether they were naive or experienced. Both groups were very reliable on the whole in judging proficiency level, although the experienced were somewhat more reliable. The experienced raters were much better at identifying correctly the source language backgrounds than the naive group. (CFM)