A Markov model for predicting performance on criterion-referenced tests is presented,. The model is expressed mathematically as a function of transition matrix, a current state vector, and a future state vector. The matrix is defined in terms of conditional probabilities, i.e., the probability of making a transition to a specific future…

Summarizing work which is part of an Army research program on Methodological Issues in the Construction of Criterion Referenced Tests, the focus of this paper is on a Bayesian model, which gives the probability of correctly classifying an examiner as a master or as a nonmaster while taking into consideration the test length and the mastery cut-off…

Effects of dimensionality on ability estimation of an adaptive test were examined using generated data in Bayesian computerized adaptive testing (CAT) simulations. Generally, increasing interdimensional difficulty association produced a slight decrease in test length and an increase in accuracy of ability estimation as assessed by root mean square…

Maximum likelihood and Bayesian procedures are presented for item selection and scoring of multidimensional adaptive tests. A demonstration with simulated response data illustrates that multidimensional adaptive testing can provide equal or higher reliabilities with fewer items than are required in one-dimensional adaptive testing. (SLD)

Five issues relative to the use of different Ordinary Least Squares (OLS) and Hierarchical Linear Modeling (HLM) models to identify effective schools and teachers were examined using data from all students in the Dallas (Texas) public schools in grade 3 in 1994 and grade 4 in 1995. OLS models using first- and second-order interactions produced…

The focus of this study is on the estimation procedures implemented in BILOG, a computer program. One purpose is to compare the item parameter estimates produced by various procedures available in BILOG. Four different models are used: the one, two, and three parameter model and a three parameter model with common guessing parameters. The results…

The use of Bayesian decision theory to solve problems in test-based decision making is discussed. Four basic decision problems are distinguished: (1) selection; (2) mastery; (3) placement; and (4) classification, the situation where each treatment has its own criterion. Each type of decision can be identified as a specific configuration of one or…

Recent work in factor analysis of categorical variables is reviewed, emphasizing a generalized least squares solution and a maximum likelihood approach. A common factor model for dichotomous items is introduced, and the estimation of factor loadings from matrices of tetracorrelations is discussed. (LMO)

Bias, standard error, and reliability of five ability estimators were evaluated using Monte Carlo estimates of the unknown conditional means and variances of the estimators. Results indicate that estimates based on Bayesian modal, expected a posteriori, and weighted likelihood estimators were reasonably unbiased with relatively small standard…

Several regression methods were examined within the framework of weighted structural regression (WSR), comparing their regression weight stability and score estimation accuracy in the presence of outlier contamination. The methods compared are: (1) ordinary least squares; (2) WSR ridge regression; (3) minimum risk regression; (4) minimum risk 2;…

The reliability of four branching algorithms commonly used in computer adaptive testing (CAT) was examined. These algorithms were: (1) maximum likelihood (MLE); (2) Bayesian; (3) modal Bayesian; and (4) crossover. Sixty-eight undergraduate college students were randomly assigned to one of the four conditions using the HyperCard-based CAT program,…

Classical statistical methods and the small enrollments in graduate departments have constrained the Graduate Record Examinations (GRE) Validity Study Service to providing only validities for single predictors. Estimates of the validity of two or more predictors, used jointly, are considered too unreliable because the corresponding prediction…

Topics concerning latent trait theory are addressed: (1) dimensionality of latent space, local independence, and item characteristic curves; (2) models--equations, parameter estimation, testing assumptions, and goodness of fit, (3) applications test developments, item bias, tailored testing and equating; and (4) advantages over classical…

The conventional method of measuring ability--based on items with assumed true parameter values obtained from a pretest--is compared to a Bayesian method that deals with the uncertainties of such items. Data from a 1987 American College Testing Program mathematics test indicate that maximum likelihood/Bayesian techniques underestimate uncertainty.…

Methods for predicting specific college course grades, based on small numbers of observations, were investigated. These methods use collateral information across potentially diverse institutions to obtain refined within-group parameter estimates. One method, referred to as pooled least squares with adjusted intercepts, assumes that slopes and…