Numerous assessments contain a mixture of multiple choice (MC) and constructed response (CR) item types and many have been found to measure more than one trait. Thus, there is a need for multidimensional dichotomous and polytomous item response theory (IRT) modeling solutions, including multidimensional linking software. For example,…

The technique of nonmetric multidimensional scaling (MDS) was applied to real item response data obtained from a multiple-choice achievement test of unknown dimensionality. The goal was to classify the 50 items into the various subtests from which they were drawn originally, the latter being unknown to the investigator. Issues addressed in the…

A new method for evaluating the content representation of a test is illustrated. Item similarity ratings were obtained from three content domain experts to assess whether ratings corresponded to item groupings specified in the test blueprint. Multidimensional scaling and cluster analysis provided substantial information about the test's content…

Alternatives to multidimensional scaling analysis of numerical relationship judgments in extracting information about students' knowledge were investigated. Undergraduate students completed a multiple choice test over knowledge of individual concepts, an essay test covering relationships among concepts, and a relationship judgment test that…

This study applies multidimensional scaling (MDS) to a knowledge organization problem involving 12 concepts in pulmonary physiology. The concept structures were derived from individual medical and veterinary students before and after focused instruction. One hundred seventy medical students and 85 veterinary medical students completed a…

Although some researchers have argued against use of the term "content validity," the ability of a test item to adequately represent the domain of knowledge tested continues to be an issue of paramount importance in test construction. The present paper reviews previous analyses of test content and proposes a new empirical method for…

Examination of a test's internal structure can be used to identify what domains or dimensions are being measured, identify relationships between the dimensions, provide evidence for hypothesized multidimensionality and test score interpretations, and identify construct-irrelevant variance. The purpose of this research is to provide a…

Formula score theory (FST) associates each multiple choice test with a linear operator and expresses all of the real functions of item response theory as linear combinations of the operator's eigenfunctions. Hard measurement problems can then often be reformulated as easier, standard mathematical problems. For example, the problem of estimating…

A procedure for interpreting multiple-discrimination indices from a multidimensional item-response theory analysis is described and demonstrated with responses of 1,635 high school students to a multiple-choice test. The procedure consists of converting discrimination parameter estimates to direction cosines and analyzing the angular distances…

The purpose of this study was to examine the nature and degree of differences in expert versus novice knowledge structures, both before and after training, when judging the similarity of multiple-choice test items within a statistics and test theory (STT) domain. Subjects were employees of the Testing Division of the New York State Department of…