This paper describes the development and implementation of an evaluation system for a pediatric residency program to meet federal grant monitoring requirements. Continuous communication and collaboration with program participants characterized this naturalistic formative evaluation. Concerns identified through initial interviews of the 15 residents and six faculty members were formulated into evaluation goals in six areas: (1) the ambulatory setting; (2) the inpatient setting; (3) subspecialty support; (4) community involvement; (5) resident evaluation; and (6) teaching evaluation. Focusing on general goals rather than individual performance, evaluation activities to collect both qualitative and quantitative data were chosen for their accuracy and feasibility in collaboration with the faculty. This participatory model provided both an effective means of evaluation and a systematic framework for program change and improvement by faculty, residents and staff. (BS)