This study was a formative evaluation of a retention program for nursing students from disadvantaged backgrounds underrepresented in the nursing workforce. The program was challenging, with multilevel interventions, some implemented sequential and some concurrently. The program team was interested to identify problematic components and address them quickly, as part of implementing a rapid cycle improvement process to ensure program success. Evaluators proposed a novel three-wave formative evaluation to meet this need. Results indicated high satisfaction with the program as well as several unintended consequences related to clinical rotations and interaction with faculty and nonparticipant students. The contribution of this study is demonstrating a practical application of an innovative evaluation approach and tool, to assist program planning and to inform remedial actions in a timely manner. Lessons learned from this study contribute to explaining the dynamic nature of unintended effects of program components in the real world and their assessment.