Since the 1993 Government Performance and Results Act, performance measurement systems based on short-term program outcomes have been increasingly used to assess the effectiveness of federal programs. This paper examines the association between program performance measures and long-term program impacts, using nine-year follow-up data from a recent large-scale, national experimental evaluation of Job Corps, the nation's largest federal job training program for disadvantaged youths. Job Corps is an important test case because it uses a comprehensive performance system that is widely emulated. We find that impacts on key outcomes are not associated with measured center performance levels. Participants in higher-performing centers had better outcomes; however, the same pattern holds for comparable controls. Thus, the performance measurement system is not achieving the goal of ranking and rewarding centers on the basis of their ability to improve participant outcomes relative to what these outcomes would have been otherwise. (Contains 3 tables, 3 figures and 9 footnotes.)