Cognitive diagnostic analyses have been advocated as methods that allow an assessment to function as a formative assessment to inform instruction. To use this approach, it is necessary to first identify the skills required for each item in the test, known as a Q-matrix. However, because the construct being tested and the underlying cognitive…

Results of using multimethod factor analysis and exploratory factor analysis for the analysis of three multitrait-multimethod matrices are compared. Results suggest that the two methods can give quite different impressions of discriminant validity. In the examples considered, the former procedure tends to support discrimination while the latter…

Methods for analyzing the multitrait-multimethod matrix are reviewed and the results of their application to a classic data set are compared. It is shown that different analysis methods can yield different validity conclusions, and that the results obtained are partly dependent on the subjective judgments of the users. (Author/RC)