By the spring of 2000, forty states had begun using student test scores to rate school performance. Twenty states have gone a step further and are attaching explicit monetary rewards or sanctions to a school's test performance. In this paper, the authors focus on accountability programs in which states measure the effectiveness of individual schools and then use those measures as the basis for providing awards, imposing sanctions, giving assistance, or identifying exemplars. They highlight an underappreciated weakness of school accountability systems--the volatility of test score measures--and explore the implications of that volatility for the design of school accountability systems. They explain and document the importance of one basic characteristic, the volatility of the measures, using data from North Carolina and California elementary schools. They then spell out the implications of that volatility for the design and use of measures of school performance. Comments by David Grissmer and Helen F. Ladd are presented. (Contains 7 figures, 7 tables and 38 notes.)