Designing a theory-based assessment with sound psychometric qualities to measure a higher-order cognitive construct is a highly desired yet challenging task for many practitioners. This paper proposes a framework for designing a theory-based assessment to measure a higher-order cognitive construct. This framework results in a modularized yet…

The aim of this paper is to present a shortened version of the SIS (support intensity scale) obtained by the application of mathematical models and instruments, adopting special algorithms based on the most recent developments in artificial adaptive systems. All the variables of SIS applied to 1,052 subjects with ID (intellectual disabilities)…

Adequacy coefficients can be derived by calculating the cosines of the angles between factors' actual and theoretically expected locations within factor space. The use of adequacy coefficients during instrument development is discussed. The development of an instrument to measure thinking interests provides a heuristic framework for the…

This paper describes a simulation that determines the adequacy of various indices as decision criteria for assessing unidimensionality. Using the sum of absolute residuals from the two-parameter latent trait model, indices were obtained that could discriminate between one latent trait and more than one latent trait. (Author/BW)

Many researchers have suggested that the main cause of item bias is the misspecification of the latent ability space. That is, items that measure multiple abilities are scored as though they are measuring a single ability. If two different groups of examinees have different underlying multidimensional ability distributions and the test items are…

Test validity is a concept that has often been ignored in the context of latent trait models and in modern test theory, particularly as it relates to computerized adaptive testing. Some considerations about the validity of a test and of a single item are proposed. This paper focuses on measures that are population-free and that will provide local…

Research related to the multiple choice test item is reported, as it is conducted by educational technologists in Japan. Sato's number of hypothetical equivalent alternatives is introduced. The based idea behind this index is that the expected uncertainty of the m events, or alternatives, be large and the number of hypothetical, equivalent…

Five comparisons were made relative to the quality of estimates of ability parameters and item calibrations obtained from the one-parameter and three-parameter logistic models. The results indicate: (1) The three-parameter model fit the test data better in all cases than did the one-parameter model. For simulation data sets, multi-factor data were…

Effects of the following methods for linking metrics on detection of differential item functioning (DIF) were compared: (1) test characteristic curve method (TCC); (2) weighted mean and sigma method; and (3) minimum chi-square method. With large samples, results were essentially the same. With small samples, TCC was most accurate. (SLD)

A general process for testing the feasibility of applying alternative mathematical or statistical models to the solution of a practical problem is presented and flowcharted. The system is used to develop a plan to compare models for test equating. The five alternative models to be considered for equating are: (1) anchor test equating using…

Hierarchical (adaptive) and linear methods of testlet construction were compared. The performance of 2,080 ninth and tenth graders on a 4-item testlet was used to predict performance on the entire test. The adaptive test was slightly superior as a predictor, but the cost of obtaining that superiority was considerable. (SLD)

A model is presented using examinee judgements in detecting ambiguous/misinterpreted items on teacher-made criterion-referenced tests. A computational example and guidelines for constructing domain categories and interpreting the indices are presented. (Author/PN)

This study investigated the convergent validity of expert-system scores for four mathematical constructed-response item formats. A five-factor model was proposed comprised of four constructed-response format factors and a Graduate Record Examinations (GRE) General Test quantitative factor. Subjects were drawn from examinees taking a single form of…

Studies carried out over a 12-year period addressed fundamental questions on the use of Rasch-based item banks. Large field tests administered in grades 3-8 of reading, mathematics, and science items, as well as standardized test results were used to explore the possible effects of many factors on item calibrations. In general, the results…

Previous research has studied the effects of different methods of item option weighting on the reliability and concurrent and predictive validity of achievement tests. Increases in reliability are generally found, but with mixed results for validity. Several methods of producing option weights, (i.e., Guttman internal and external weights and…