The authors examined how the regular use of Monte Carlo Quizzes, learning logs, or non-random quizzes impacted student completion of assigned reading and preparedness for class. A questionnaire was used to gather data on student perceptions and self-reported reading behavior in response to each of the methods over the course of a semester. Results…

A major issue in the utilization of covariance structure analysis is model fit evaluation. Recent years have witnessed increasing interest in various test statistics and so-called fit indexes, most of which are actually based on or closely related to F[subscript 0], a measure of model fit in the population. This study aims to provide a systematic…

Score limitation at the top of a scale is commonly termed "ceiling effect." Ceiling effects can lead to serious artifactual parameter estimates in most data analysis. This study examines the consequences of ceiling effects in longitudinal data analysis and investigates several methods of dealing with ceiling effects through Monte Carlo simulations…

In Ramsay-curve item response theory (RC-IRT), the latent variable distribution is estimated simultaneously with the item parameters. In extant Monte Carlo evaluations of RC-IRT, the item response function (IRF) used to fit the data is the same one used to generate the data. The present simulation study examines RC-IRT when the IRF is imperfectly…

This research examines the utility of the s-x[superscript 2] statistic proposed by Orlando and Thissen (2000) in evaluating item fit for multidimensional item response models. Monte Carlo simulation was conducted to investigate both the Type I error and statistical power of this fit statistic in analyzing two kinds of multidimensional test…

Reliability generalization studies have provided estimates of the mean reliability coefficients and examined factors that explain the variability in the reliability estimates across studies for many different tests and measures. Different authors have used different data analyses to do such meta-analyses, and little research has addressed whether…

Cognitive diagnostic models (CDMs) model the probability of correctly answering an item as a function of an examinee's attribute mastery pattern. Because estimation of the mastery pattern involves more than a continuous measure of ability, reliability concepts introduced by classical test theory and item response theory do not apply. The cognitive…

One major challenge in using group projects to assess student learning is accounting for the differences of contribution among group members so that the mark assigned to each individual actually reflects their performance. This research addresses the validity of grading group projects by evaluating different methods that derive individualized…

This article presents a new algorithm for computerized adaptive testing (CAT) when content constraints are present. The algorithm is based on shadow CAT methodology to meet content constraints but applies Monte Carlo methods and provides the following advantages over shadow CAT: (a) lower maximum item exposure rates, (b) higher utilization of the…

Tests of model misfit are often performed to validate the use of a particular model in item response theory. Douglas and Cohen (2001) introduced a general nonparametric approach for detecting misfit under the two-parameter logistic model. However, the statistical properties of their approach, and empirical comparisons to other methods, have not…

The growth curve model has been a useful tool for the analysis of repeated measures data. However, it is designed for an aggregate-sample analysis based on the assumption that the entire sample of respondents are from a single homogenous population. Thus, this method may not be suitable when heterogeneous subgroups exist in the population with…

Examined the Type I error rate and the power of the Stout T procedure (DIMTEST) (W. Stout, 19987, 1990) and the Holland-Rosenbaum procedure (P. Holland and P. Rosenbaum, 1986) for normal and nonnormal data sets through a Monte Carlo study. Both procedures performed adequately under some conditions, but the Stout T procedure showed adequate power…

In current psychological research, the analysis of data from computer-based assessments or experiments is often confined to accuracy scores. Response times, although being an important source of additional information, are either neglected or analyzed separately. In this article, a new model is developed that allows the simultaneous analysis of…

The originally proposed multivariate meta-analysis approach for correlation matrices--analyze Pearson correlations, with each study's observed correlations replacing their population counterparts in its conditional-covariance matrix--performs poorly. Two refinements are considered: Analyze Fisher Z-transformed correlations, and substitute better…

This article introduces a bootstrap generalization to the Modified Parallel Analysis (MPA) method of test dimensionality assessment using factor analysis. This methodology, based on the use of Marginal Maximum Likelihood nonlinear factor analysis, provides for the calculation of a test statistic based on a parametric bootstrap using the MPA…