The methodology of P. E. Holland and P. R. Rosenbaum (1986) to assess unidimensionality of binary data is outlined and illustrated through a simulation with 36 items for 2,000 examinees. How to interpret the results is discussed. (SLD)

The extent to which standardized regression coefficients (beta values) can be used to determine the importance of a variable in an equation was explored. The beta value and the part correlation coefficient--also called the semi-partial correlation coefficient and reported in squared form as the incremental "r squared"--were compared for…

Standard procedures for estimating item parameters in item response theory ignore collateral information that may be available about examinees. Circumstances under which collateral information about examinees may be used to make inferences about item parameters more precise and circumstances under which collateral information must be used are…

For anchor test equating, 3 linear observed score methods are derived for populations differing in ability. Each version requires that the correlations of the tests with the selection variable be known. Five sets of assumptions are made for each model--yielding 15 methods--which are then related to existing methods. (SLD)

This paper explores a logistic regression procedure for estimating item parameters in the Rasch model and testing the hypothesis of item parameter invariance across several groups/populations. Rather than using item responses directly, the procedure relies on "pseudo-paired comparisons" (PC) statistics defined over all possible pairs of…

A two-stage process that considers the multidimensionality of tests under the framework of unidimensional item response theory is described and evaluated. Findings from a simulation indicate that gains in estimation robustness and score interpretation are possible with almost no sacrifice of goodness of fit when using this two-stage approach. (SLD)

This paper discusses standard error of measurement (SEM), the amount of variation or spread in the measurement errors for a test, and gives information needed to interpret test scores using SEMs. SEMs at various score levels should be used in calculating score bands rather than a single SEM value. (SLD)

Scaling methodologies used in analysis of National Assessment of Educational Progress (NAEP) surveys since 1984 are reviewed. The plausible values methodology developed for NAEP scale-score analyses is described in the contexts of item response theory and average response method scaling. Current NAEP research in scaling is discussed. (SLD)

Forming the matching variable for the Mantel-Haenszel differential item functioning (DIF) procedure through use of the total score as the matching variable (thin) and forming the matching variable by pooling total score levels (thick) were compared in a Monte Carlo study. Reasons thick matching is superior are discussed. (SLD)

Analyzes effect sizes in tails of distribution of scores in Feingold's study of joint effects of gender differences in mean and variability on 28 cognitive-ability scales. Effect sizes are smaller than Feingold assumed. Evaluates joint effect of gender differences by number of males and females in extreme score ranges. (SLD)

This paper illustrates how, in the item-response theory framework, collateral information about test items can augment or replace examinee responses when linking or equating new tests to established scales, using data from the Pre-Professional Skills Test for approximately 40,000 examinees. Collateral information can predict item operating…

Procedures used to map the achievement levels, expressed as total expected scores on the full set of items presented to a grade, onto the Mathematics Composite of the 1990 National Assessment of Educational Progress (NAEP) are described. The Composite, defined as a weighted average of subdomain scores, provides a global measure of mathematics…

The conventional method of measuring ability--based on items with assumed true parameter values obtained from a pretest--is compared to a Bayesian method that deals with the uncertainties of such items. Data from a 1987 American College Testing Program mathematics test indicate that maximum likelihood/Bayesian techniques underestimate uncertainty.…

The shortcomings of the conventional way of using and interpreting multiple-choice tests are summarized. Some theories and methodologies that can be applied for better use multiple-choice test items are described. Empirical facts are introduced to support the theoretical observations. New strategies are proposed that will reduce "noise"…

The phenomenon of simultaneous differential item functioning (DIF) amplification and cancellation and the role of the SIBTEST computer program in detecting it were studied. A variety of simulated test data was generated for this purpose. In addition, the following real test data were used: (1) American College Testing program data for 2,115 males…