When constructed response test items are scored by more than one rater, the repeated ratings allow for the consideration of individual rater bias and variability in estimating student proficiency. Several hierarchical models based on item response theory have been introduced to model such effects. In this article, the authors demonstrate how these…

According to Kenny and McCoach (2003), chi-square tests of structural equation models produce inflated Type I error rates when the degrees of freedom increase. So far, the amount of this bias in large models has not been quantified. In a Monte Carlo study of confirmatory factor models with a range of 48 to 960 degrees of freedom it was found that…

The impact of outliers on Cronbach's coefficient [alpha] has not been documented in the psychometric or statistical literature. This is an important gap because coefficient [alpha] is the most widely used measurement statistic in all of the social, educational, and health sciences. The impact of outliers on coefficient [alpha] is investigated for…

The authors of this article examine the analytic and reporting features of research articles cited in "Studying Teacher Education: The Report of the AERA Panel on Research and Teacher Education" (Cochran-Smith & Zeichner, 2005b) that used quantitative reporting practices. Their purpose was to help to identify reporting practices that can be…

The purpose of this ITEMS module is to provide an introduction to Markov chain Monte Carlo (MCMC) estimation for item response models. A brief description of Bayesian inference is followed by an overview of the various facets of MCMC algorithms, including discussion of prior specification, sampling procedures, and methods for evaluating chain…

Two Monte Carlo studies were conducted to examine the sensitivity of goodness of fit indexes to lack of measurement invariance at 3 commonly tested levels: factor loadings, intercepts, and residual variances. Standardized root mean square residual (SRMR) appears to be more sensitive to lack of invariance in factor loadings than in intercepts or…

The jackknife and bootstrap methods are becoming more popular in research. Although the two approaches have similar goals and employ similar strategies, information is lacking with regard to the comparability of their results. This study systematically investigated the issue for a canonical correlation analysis, using data from four random samples…

When multiple linear regression is used to develop a prediction model, sample size must be large enough to ensure stable coefficients. If the derivation sample size is inadequate, the model may not predict well for future subjects. The precision efficacy analysis for regression (PEAR) method uses a cross- validity approach to select sample sizes…

This Monte Carlo investigation provides some possible solutions to problems related to choosing multiple-comparison methods that maximize true rejections and minimize false ones. It has been argued that the traditional Bonferroni approach to multiple comparisons, which satisfies the statistician's family-wise Type I error concerns, could be…

A Monte Carlo investigation of Markov chain matrices was conducted to create empirical distributions for two statistics created from the transition matrices. Curve fitting techniques developed by Karl Pearson were used to deduce if theoretical equations could be fit to the two sets of distributions. The set of distributions which describe the…

Computer sampling from a multivariate normal spherical population was used to evaluate Type I error rates for a test of P = I based on Fisher's tanh(sup minus 1) variance stabilizing transformation of the correlation coefficient. (Author/TJH)

Outlines costs and benefits associated with different cross-validation strategies; in particular the way in which the study design affects the cost and benefits of different types of cross-validation. Suggests that the choice between empirical estimation methods and formula estimates involves a trade-off between accuracy and simplicity. (JAC)

Uses Monte Carlo methods to assess the relative power of the paired samples t test and Wilcoxon's signed-ranks test under 10 population shapes. Concludes that, insofar as these two statistics are concerned, the often-repeated claim that parametric tests are more powerful than nonparametric tests is not justified. (Author/CB)

Critical examination is made of the recent controversy over the value of Monte Carlo techniques in nonmetric multidimensional scaling procedures. The case is presented that the major relevance of Monte Carlo studies is not for the local minima problem but for the meaningfulness of the obtained solutions. (Author)

The purpose of this study was to examine the recovery of item parameters in simulated Automatic Item Generation (AIG) conditions, using Markov chain Monte Carlo (MCMC) estimation methods to attempt to recover the generating distributions. To do this, variability in item and ability parameters was manipulated. Realistic AIG conditions were…