The Air Force Occupational Measurement Center conducts task-based occupational surveys of Air Force specialties that include supervisor ratings on recommended training emphasis for entry-level airmen. Priorities are input to the Instructional System Development training model, which guides the development and revision of specialty training courses. For 20 percent of specialties, training emphasis ratings have been subject to poor interrater agreement. Data may contain conflicting rating policies within a specialty. To develop a methodology for identifying multiple rating policies in such data, this research investigates: (1) the variation in interrater agreement with respect to sample size; and (2) the multiple rating policy hypothesis via modified REXALL analysis, cluster analysis, and factor analysis. Agreement is found to vary within and across sample sizes, and a minimum of 55 raters is recommended. REXALL analyses are inconclusive with respect to confirming the presence or absence of multiple rating policies. Results indicate that samples of training emphasis ratings are less complex than expected. REXALL analyses are recommended for single ladder specialties; principal components factors analysis with VARIMAX rotation is recommended for multiple factors--extracting one and then multiple factors as appropriate. Interpretation of these results can be enhanced with CODAP auxiliary programs. (LMO)