This study was undertaken to compare non-metric multidimensional scaling (MDS) and factor analysis (FA) as means of assessing dimensionality in relation to item response theory (IRT). FA assesses correlation matrices, while MDS performs an analysis of proximity measures. Seven data sets were generated; each differed from the others with respect to…

The rationale behind analysis of variance (including analysis of covariance and multiple analyses of variance and covariance) methods is reviewed, and unplanned and planned methods of evaluating differences between means are briefly described. Two advantages of using planned or a priori tests over unplanned or post hoc tests are presented. In…

Transformations or equating of raw test scores on two or more forms of the same test are made interchangeable by empirical procedures deriving the standard error of an equipercentile equating for four different situations. Some numerical results are checked by Monte Carlo methods. Numerical standard errors are computed for two sets of real data.…

This paper describes how work by the United States Office of Personnel Management on the generalizability of employment test validities led to the development of a widely applicable meta-analysis method. The method focuses strongly on estimating the true variance of study correlations and effect size. This validity generalization procedure has…

Introductory statistics texts give extensive coverage to two-sided inferences in hypothesis testing, interval estimation, and one-sided hypothesis tests. Very few discuss the possibility of one-sided interval estimation at all. Even fewer do so in any detail. Two of the business statistics texts we reviewed mentioned the possibility of dividing…

The standard error of measurement (SEM) is a measure of the inconsistency in the scores of a particular group of test-takers. It is largest for test-takers with scores ranging in the 50 percent correct bracket; with nearly perfect scores, it is smaller. On tests used to make pass/fail decisions, the test-takers' scores tend to cluster in the range…

The test performance of students who took the Scholastic Aptitude Test (SAT) only once as juniors was contrasted with students who took the test as juniors and again as seniors. Estimates of expected test performance on a common initial administration in the junior year were derived from separate equating sections and background variables.…

A general framework for making mastery/nonmastery decisions based on multivariate test data is described in this study. Over all, mastery is granted (or denied) if the posterior expected loss associated with such action is smaller than the one incurred by the denial (or grant) of mastery. An explicit form for the cutting contour which separates…

In an invitational address to the Victorian Institute of Educational Research, the author discussed Bayesian theory and its relationship to the design and construction of tailored or adaptive tests. Bayesian thinking involves recognizing the role of prior probabilities and using these probabilities in combination with new data to arrive at future…

Written by an ex-Latin teacher, short-cuts to analyzing test results for the non-mathematical teacher are provided. Discussions are given of item analysis (item analysis by a show of hands, standards for test items: success, standards for test items: discrimination, and the second stage of item analysis. The standard error is then presented (the…

The standard error of measurement as a means for estimating the margin of error that should be allowed for in test scores is discussed. The true score measures the performance that is characteristic of the person tested; the variations, plus and minus, around the true score describe a characteristic of the test. When the standard deviation is used…

A simple, rigorous, small-sample statistical technique is described for testing the hypothesis that two sets of measurements differ only because of errors of measurement and because of differing origins and units of measurement. (Author)

Procedures for establishing standards and determining the number of items needed in criterion-referenced measures are reviewed. The discussion of setting a passing score is organized around five factors: performance of others, item content, educational consequences, psychological and financial costs, and measurement error. Classical test theory,…

Because of limitations in simple gain scores, pychometrists have proposed alternate methods for measuring change, two of which are residualized difference and base-free change. This paper provides large sample empirical estimates of the reliability these change measures. It checks theoretical predictions derived from inequalities involving all…

Psychometricians working in factor analysis and econometricians working in regression with measurement error in all variables are both interested in the rank of dispersion matrices under variation of diagonal elements. This paper reviews both fields; points out various small errors; and presents a methodological comparision of factor analysis and…