Generalizability theory extends previous methods of estimating the reliability of rating instruments such that one can estimate the precision of a measurement system for differentiating among students, scales, or any other important dimension. In this study, generalizability theory is applied to faculty ratings of junior and senior dental students on five broadly defined scales. Utilizing two observers, the measurement system differentiates accurately among students, however, interpretation of inter-scale differences for a given student are less dependable. The methodology demonstrated is recommended for use in pilot testing rating instruments for evaluating clinical performance. (Author/MV)