A large urban northwest school district is engaged in the construction of a central accountability policy that depends on school-based implementation. As part of this effort, the district has implemented a standards-based exit policy. Each school has identified students as either meeting grade-level standards or not meeting these standards. This paper describes an investigation of the application of the policy at various schools and its impact on the students identified as not having the skills to exit a grade. The district has identified 534 fourth graders without the skills to exit. Issues of disproportionately low achievement among minority groups, particularly African Americans, are of central concern in this district. A study examined: (1) the validity of teacher judgments; (2) the relationship between teacher judgments based on classroom evidence and district and state evidence; (3) error rate disproportionalities; and (4) error rates by school. The validity of teacher judgment was influenced by differences between schools in the rates of implementation and use of classroom evidence, although professional training in assessment was creating some uniformity in classroom evidence. The relationships between classroom assessments and state assessments indicate that teachers may rely on demonstrated computational skills as evidence of student achievement, based on the use of the mandated state test. The most striking finding was the disproportionality of error rate by ethnic group, gender, and socioeconomic status. Implications of these findings for educational equity in this urban district are discussed. (Contains 29 references.) (SLD)