This paper assesses the credibility of a single total instrument score and various logical sub-scores derived from a series of summative judgments about the quality of teaching performance. The objectives were to compare the generalizability of alternative Teacher Performance Assessment Instrument (TPAI) scores, to compare the dependability of decisions which could be made with the scores, and to compare the relationship of the scores with learner achievement. Measures were made of teacher performance using the revised version of the TPAI. Learner ability was assessed with the Group Assessment of Logical Thinking in order to equate classes. Learner achievement was assessed with the Middle Grades Integrated Process Skill Test. Results show that: (1) the aggregation of summative judgments used in the TPAI scoring can be a valid and reliable procedure; (2) intermediate levels of scoring such as the TPAI competencies are more desirable than total instrument scores; (3) the total is a more reliable, but less valid, indicator of effectiveness; and (4) validity and dependability coefficients are adequate evidence to support the validity and reliability of the competency scores. Caution should be excerised in inferring causality of these teacher behaviors or learner outcomes based on these results. (PN)