The aims of this study were: (1) to review and discuss alternatives for evaluating members of a set of scores which may be interpreted independently as contributing to a composite score, or as a member of a profile and (2) to examine multivariate generalizability theory, as suggested by Joe and Woodward, applied to student course/instructor ratings by analyzing course evaluation questionnaires. Described are the procedures which were applied to data obtained from seven samples on two campuses to derive multivariate generalizability coefficients associated with item score patterns that maximally discriminate between courses and instructors. Analysis results for the two basic design figurations (instructors nested within courses and courses nested within instructors), and the results of split halves analyses, used to ascertain the stability of composite weights across samples, are discussed and displayed in tabular form. Discussion of the results of the study stresses the limitation of the Joe and Woodward procedure since it was found that weights assigned to different items were highly variable across split-halves analyses. It is suggested that the stability of results across samples will be greatest when the number of variables is small or the sample size very large. (Author/AEF)