As part of a new teacher evaluation program initiated by the local school board, the Charleston County School District (South Carolina) adopted the Assessments of Performance in Teaching (APT) as a major evaluation tool to assess the teaching performance of annual contract teachers. Since evaluation procedures can ultimately lead to teacher dismissal, it was incumbent upon the district staff to ensure the appropriateness of the APT and its technical quality for a population of teachers wider than those for whom the instrument was designed. A study was conducted on approximately 250 teachers to examine the inter-observation and inter-rater reliability of the APT for various groups of teachers: special education teachers, Chapter 1 teachers, elementary, middle and high school teachers, black teachers and white teachers. Agreement indices were calculated for individual items to identify teacher behaviors which reduced reliability and for which observers need additional training and practice. Other local concerns addressed by the study focused on differences in the ratings of principals versus district staff and ratings of observers evaluating teachers within their own field of certification versus observers evaluating teachers in fields outside their own. (Author)