This paper is meant to help designers of educational accountability systems think more critically about the questions they want to ask schools and the kinds of analyses that best answer these questions. There are two essential questions in school evaluation: how good is this school, and is it getting better? Each of these questions then can be subdivided into achievement level status and achievement level change. These four questions can be examined in a two-by-two matrix that results in quadrants with questions: (1) what is the achievement level of students in this school (quadrant A, question 1a); (2) is the achievement level of this school improving (quadrant B, question 2a); (3) is this an effective school (quadrant c, question 1b); and (4) is this school becoming more effective (quadrant d, question 2b). Developers of accountability systems must consider the implications of these questions for their systems. (Contains 4 exhibits and 19 references.) (SLD)