The effectiveness of two iterative methods of item response theory (IRT) item bias detection was examined in a simulation study. A modified form of the iterative item parameter linking method of F. Drasgow and an adaptation of the test purification procedure of F. M. Lord were compared. (SLD)

The polytomous unidimensional Rasch model with equidistant scoring (rating scale model) is extended so that two parameters are linearly decomposed into certain basic parameters. A conditional maximum likelihood estimation procedure and a likelihood ratio test are presented in the context of the extended model (linear rating scale model). (SLD)

A modification of a test of the equality of nonindependent alpha reliability coefficients is proposed. It avoids the limitation that the product of the number of test parts times the number of subjects be quite large. Monte Carlo studies indicate that this test can be used in comparing interrater reliabilities. (SLD)

Three correlation matrices (PEARSON, POLYCHORIC, and TOBIT) were used to obtain reliability estimates on ordered polytomous variable models. A Monte Carlo study with different levels of variable asymmetry and 400 sample correlation matrices demonstrated that the PEARSON matrix did not perform as well as did the other 2 matrices. (SLD)

Describes an automated scoring algorithm for a computer-based simulation examination of physicians' patient-management skills. Results with 280 medical students show that scores produced using this algorithm are highly correlated to actual clinician ratings. Scores were also effective in discriminating between case performance judged passing or…

The effectiveness of alternative procedures to correct for rater leniency/stringency effects was studied when true scores were known. Ordinary least squares, weighted least squares, and imputation of the missing data consistently outperformed averaging the observed ratings; and the imputation technique was superior to the least squares methods.…

Performance rating systems frequently use multiple raters in order to improve the reliability of ratings. However, unless all candidates are rated by the same raters, some candidates will be at an unfair advantage or disadvantage solely because they were rated by more stringent or lenient raters. To obtain fair and accurate evaluations of…