A study was conducted that was designed to determine if experienced ESL (English as a second language) teachers, working in pairs, can reach the same judgments regarding the oral proficiency of non-native speakers of English and to determine the reliability of such judgments. In addition, the question of whether different sets of judges rate differently is posed. Finally, the study was designed to determine the relative weight given to each component category in predicting the overall proficiency score. Five judges, all graduate students in linguistics who had taught ESL for at least one year, were randomly paired together to form six groups. The 98 subjects were foreign students referred to the University of Iowa for evaluation. The judges were required to rate speakers in five categories of speaking proficiency: (1) pronunciation, (2) grammar; (3) vocabulary; (4) fluency; and (5) auditory comprehension. The results showed that although there were differences in the ratings among the individual judges, an average of the two judges' ratings served as a good estimate of the true rating. The ratings on the listening comprehension, pronunciation, fluency, and overall proficiency scales did not differ from group to group. A significant difference was found, however, for the grammar scale. The overall scale appears to be a composite of the other four scales and, therefore, is the best scale of measurement for oral proficiency. (CFM)