Contributing to a competency model for engineering thermodynamics, we investigate the empirical competency structure of our exams in an attempt to answer the question: Do we test the competencies we want to convey to our students? We demonstrate that thermodynamic modeling and mathematical solution emerge as significant dimensions of thermodynamic…

The purpose of the present investigation is to examine the influence of sample size (N) and model complexity on a set of 23 goodness-of-fit (GOF) indices, including those typically used in confirmatory factor analysis. The focus was on two potential problems in assessing GOF: (1) some fit indices are substantially influenced by N so that tests of…

Real test data of unknown structure were analyzed using both a unidimensional and a multidimensional latent trait model in an attempt to determine the underlying components of the test. The models used were the three-parameter logistic model and a multidimensional extension of the two-parameter logistic model. The basic design for the analysis of…

In most item response theory models a particular mathematical form is assumed for all item characteristic curves, e.g., a logistic function. It could be desirable, however, to estimate the shape of the item characteristic curves without prior restrictive assumptions about its mathematical form. We have developed a practical method of estimating…

Qualitative evidence for the operation of partial knowledge is given by two findings. First, performance when second and subsequent choices are made is above the chance level. Second, it is positively related to first choice performance. A number of theories incorporating partial knowledge are compared quantitatively. (Author/LMO)

A study was undertaken to develop guidelines for the interpretation of the parameters of three multidimensional item response theory models and to determine the relationship between the parameters and traditional concepts of item difficulty and discrimination. The three models considered were multidimensional extensions of the one-, two-, and…

The appropriateness of the use of the standardized residual (SR) to assess congruence between sample test item responses and the one parameter latent trait (Rasch) item characteristic curve is investigated. Latent trait theory is reviewed, as well as theory of the SR, the apparent error in calculating the expected distribution of the SR, and…

A study was conducted to compare the quality of the item parameter estimates obtained from the ANCILLES and LOGIST estimation procedures using goodness of fit as a criterion. Statistics used to compare the fit included a chi-square statistic and a mean square deviation statistic. Other analyses performed included comparisons of the distributions…

A strategy for overcoming problems with the Rasch model's inability to handle missing data involves a pairwise algorithm which manipulates the data matrix to separate out the information needed for the estimation of item difficulty parameters in a test. The method of estimation compares two or three items at a time, separating out the ability…

An explanation is given of what is meant by "sample-free" item calibration and by "item-free" person measurement as these terms are applied to the one-parameter logistic test theory model of Georg Rasch. When the difficulty of an item is calibrated separately for two different samples the results may differ; but, according the…

The analysis of fit of data to a measurement model for graded responses is described. The model is an extension of Rasch's dichotomous model to formats which provide more than two levels of response to items. The model contains one parameter for each person and one parameter for each "step" in an item. A dichotomously-scored item…

Problems and procedures in assessing and obtaining fit of data to the Rasch model are treated and assumptions embodied in the Rasch model are made explicit. It is concluded that statistical tests are needed which are sensitive to deviations so that more than one item parameter would be needed for each item, and more than one person parameter would…

Issues of concern to test developers interested in applying the Rasch model are discussed. The current state of the art, recommendations for use of the model, further needs, and controversies are described for the three stages of test construction: (1) definition of the content of the test and item writing; (2) item analysis; and (3) test…

A method for analyzing test-item responses is proposed to examine differential item functioning (DIF) in multiple-choice items within the latent class framework. Different models for detection of DIF are formulated, defining the subgroup as a latent variable. An efficient estimation method is described and illustrated. (SLD)

The efficacy of employing subsets of items from a calibrated item pool to estimate the Rasch model person parameters was investigated. Specifically, the degree of invariance of Rasch model ability-parameter estimates was examined across differing collections of simulated items. The ability-parameter estimates were obtained from a simulation of…