Monte Carlo methods were used to investigate the effect of misspecification of the second level in a two-level hierarchical linear model (HLM). Sample composition, heterogeneity of the group size, level of intraclass correlation, and correlation between second-level predictors were manipulated. Each of 20 generated data sets was analyzed nine…

The benefits of item response theory (IRT) will only accrue to a testing program to the extent that model assumptions are met. Obtaining accurate item parameter estimates is a critical first step. However, the sample sizes required for stable parameter estimation are often difficult to obtain in practice, particularly for the more complex models.…

Large sample multivariate methods for estimating and comparing effect sizes across independent samples within a single study are presented. Procedures for pooling treatment effects are provided to allow determination of overall study effect sizes for treatments with similar effects prior to implementation of meta-analyses. (TJH)

A selection model for meta-analysis is proposed that models the selection process and corrects for the consequences of selection by publication on estimates of the mean and variance of the effect parameters. Simulation studies show that the model substantially reduces bias when the model specification is correct. (SLD)

Using simulation, the effect that misspecification of elements in the weight matrix has on estimates of basic parameters of the linear logistic test model was studied. Results indicate that, because specifying elements of the weight matrix is a subjective process, it must be done with great care. (SLD)

Accuracy of item parameter estimates is a critical concern for any application of item response theory (IRT). However, the necessary sample sizes are often difficult to obtain in practice, particularly for the more complex models. A promising avenue of research concerns modified item response models. This study both replicates and improves on an…

The use of two alternative item response theory (IRT) estimation models in the scaling and equating of the Test of English as a Foreign Language (TOEFL) was explored; and item scaling and test equating results based on these models were compared with results based on the three-parameter (3PL) model currently being used with the TOEFL. Models were…

This study investigated the robustness of the James second-order test (James 1951; Wilcox, 1989) and the univariate F test under a two-factor fixed-effect analysis of variance (ANOVA) model in which cell variances were heterogeneous and/or distributions were nonnormal. With computer-simulated data, Type I error rates and statistical power for the…