Several multivariate statistical methodologies have been proposed to ensure objective and quantitative evaluation of the multitrait-multimethod matrix. The paper examines the performance of confirmatory factor analysis and covariance component models. It is shown, both empirically and formally, that confirmatory factor analysis is not a reliable…

A nominal response model-based computerized adaptive testing procedure (nominal CAT) was implemented using simulated data. Ability estimates from the nominal CAT were compared to those from a CAT based upon the three-parameter logistic model (3PL CAT). Furthermore, estimates from both CAT procedures were compared with the known true abilities used…

Six editions of Scholastic Aptitude Test-Mathematical (SAT-M) were factor analyzed using confirmatory and exploratory methods. Confirmatory factor analyses (using the LISREL VI program) were conducted on correlation matrices among item parcels--sums of scores on a small subset of items. Item parcels were constructed to yield correlation matrices…

Factor analysis is the traditional method for studying the dimensionality of test data. However, under common conditions, the factor analysis of tetrachoric correlations does not recover the underlying structure of dichotomous data. The purpose of this paper is to demonstrate that the factor analyses of tetrachoric correlations is unlikely to…

Statistical methodologists have sometimes criticized the use of conventional statistics in meta-analysis, and in recent years a number of them have advocated the use of a special new statistical methodology for research synthesis. An examination of recent books describing this methodology shows that it is seriously limited in its applicability to…

This paper examines the pattern of diffusion in the academic literatures of the sciences, social sciences, and arts and humanities based on citations. An examination of the citations of articles in the Science Citation Index, Social Science Citation Index, and Arts and Humanities Citation Index from a given year to the year in which the cited…

This study examined how four commonly used test equating procedures (linear, equipercentile, Rasch Model, and three-parameter) would respond to situations in which the properties or the two tests being equated were different. Data for two tests plus an external anchor test were generated from a three parameter model in which mean test differences…

Classroom researchers are frequently urged to provide evidence of the reliability of their data. In the case of observational data, three approaches to this have emerged: observer agreement, generalizability theory, and measurement error. Generalizability theory provides the most powerful approach given an adequate data collection design, but…

In 1972, the Oregon State Department of Education changed its standards relating to graduation requirements after a survey revealed concern that the high school diploma had lost its credibility. Parkrose School District developed a goal-based curriculum and measurement program to meet the new requirements. From the program's inception, District…

This study introduced a probabilistic model utilizing item response theory (IRT) for dealing with a variety of misconceptions. The model can be used for evaluating the transition behavior of error types, advancement of learning stages, or the stability and persistence of particular misconceptions. Moreover, it apparently can be used for relating…

The purpose of this research study was to assess item response model-test data fit using residuals. First, a comparison of raw and standardized residuals for describing model-test data fit was carried out. Second, hypotheses concerning the relationship between residual sizes and several item characteristics were studied. The analyses with…

Comparisons were made of three Angoff Design V linear equating methods (two forms equated to a common test, two forms predicted by a common test, or two forms used to predict a common test) and Tucker's and R. Levine's linear methods, under common item linear equating with non-equivalent populations. Forms of a professional certification test…

A general approach is proposed for modeling the structure of a two-way contingency table, and for drawing inferences about the marginal and interaction effects, cell parameters, and conditional probabilities. The prior distribution expresses uncertainty in a simple reduced model, in particular the independence model. The posterior estimates of the…

Ways in which the Statistical Package for the Social Sciences (SPSS) can be used to perform some Rasch analyses are described in detail. It is shown how SPSS and a set of item calibrations can be used to estimate person abilities, standard errors of measurement, test characteristic curve, test information curve, classification consistency on a…

Some diverse fields in which the effects of mental representations have been noticed are first discussed, largely through illustrations of differing problem-solving experiences. Then the LAKOS Project (Logical Analysis of Cognitive Organizational Structures) at the University of Osnabruck is described. A central concern is the specification of a…