Because analyses of reaction-time data are sensitive to aberrant observations and violations of statistical assumptions, a new approach is suggested. In this empirical approach, one applies the same criteria to the problem of selecting a statistical method as one uses to select among alternative experimental procedures. Six criteria are presented…

Previous research has studied the effects of different methods of item option weighting on the reliability and concurrent and predictive validity of achievement tests. Increases in reliability are generally found, but with mixed results for validity. Several methods of producing option weights, (i.e., Guttman internal and external weights and…

Research using Social Security data to measure the effects of government manpower training programs on the earnings of participants is discussed. Previous studies employed a longitudinal set of Social Security summary earnings records of over 50,000 individuals who participated in Manpower Development and Training Act (MDTA) institutional training…

Mathematical models of judgment policy are compared in evaluating 625 Cadet Performance Reports during a single rating cycle. Linear and nonlinear regressions are used to describe the cues most important in determining overall ratings. In addition, performance rating policy is compared with stated policy for each of forty cadet raters. In general,…

The purpose of this study was to establish score equivalencies between the College Board Scholastic Aptitude Test (SAT) and its Spanish-language equivalent, the College Board Prueba de Aptitud Academica (PAA). The method of the study involved two phases: the selection of test items equally appropriate for Spanish- and English-speaking students for…

Failure to consider errors of measurement when using partial correlation or analysis of covariance techniques can result in erroneous conclusions. Certain aspects of this problem are discussed and particular attention is given to issues raised in a recent article by Brewar, Campbell, and Crano. (Author)

The effectiveness of alternative procedures to correct for rater leniency/stringency effects was studied when true scores were known. Ordinary least squares, weighted least squares, and imputation of the missing data consistently outperformed averaging the observed ratings; and the imputation technique was superior to the least squares methods.…

Results of empirical validation studies of six statistical models available for indirectly estimating the need for alcohol, drug abuse, and mental health services across a large geographic area are presented. Models were evaluated using Colorado Social Health Survey data for 4,745 adults from 48 subareas of Colorado. (SLD)

The conventional method of measuring ability--based on items with assumed true parameter values obtained from a pretest--is compared to a Bayesian method that deals with the uncertainties of such items. Data from a 1987 American College Testing Program mathematics test indicate that maximum likelihood/Bayesian techniques underestimate uncertainty.…

The adequacy of several approaches to estimation of proficiency distributions for the Trial State Assessment (TSA) in eighth grade mathematics of the National Assessment of Educational Progress was examined. These approaches are more restrictive than the estimation procedures originally used, with the same kind of plausible-values approach that…

Methods for predicting specific college course grades, based on small numbers of observations, were investigated. These methods use collateral information across potentially diverse institutions to obtain refined within-group parameter estimates. One method, referred to as pooled least squares with adjusted intercepts, assumes that slopes and…

This paper presents an introduction to the use of latent trait models for the estimation of domain scores. It was shown that these models provided an advantage over classical test theory and binomial error models in that unbiased estimates of true domain scores could be obtained even when items were not randomly selected from a universe of items.…

This paper reports the results of a Monte Carlo investigation of Type I errors in the single group repeated measures design where multiple measures are collected from each observational unit at each measurement occasion. The Type I error of three multivariate tests were examined. These were the doubly multivariate F test, the multivariate mixed…

A study was conducted to compare tailored testing procedures based on a Bayesian ability estimation technique and on a maximum likelihood ability estimation technique. The Bayesian tailored testing procedure selected items so as to minimize the posterior variance of the ability estimate distribution, while the maximum likelihood tailored testing…

Alternative mathematical models for scoring and decision making with criterion referenced tests are described, especially as they concern appropriate test length and methods of establishing statistically valid cutting scores. Several of these approaches are reviewed and compared on formal-analytic and empirical grounds: (1) Block's approach to…