A large pediatric residency program conducted an extensive analysis of the reliability and validity of the rating forms used to evaluate the pediatric residents enrolled in the program. Data indicate that although the reliability of individual ratings is very low, several factors achieved acceptable levels of reliability when aggregated. (Author/LMO)