This study represented a first attempt to evaluate the impact of local item dependence (LID) for Item Response Theory (IRT) scoring in computerized adaptive testing (CAT). The most basic CAT design and a simplified design for simulating CAT item pools with varying degrees of LID were applied. A data generation method that allows the LID among…

When multiple regression is used to develop a prediction model, sample size must be large enough to ensure stable coefficients. If sample size is inadequate, the model may not predict well in future samples. Unfortunately, there are problems and contradictions among the various sample size methods in regression. For example, how does one reconcile…

J. Barnette and J. McLean (1996) proposed a method of controlling Type I error in pairwise multiple comparisons after a significant omnibus F test. This procedure, called Alpha-Max, is based on a sequential cumulative probability accounting procedure in line with Bonferroni inequality. A missing element in the discussion of Alpha-Max was the…

The Mantel Haenszel (MH; 1959) approach of Holland and Thayer (1988) is a well-established method for assessing differential item functioning (DIF). The formula for the variance of the MH DIF statistic is based on work by Phillips and Holland (1987) and Robins, Breslow, and Greenland (1986). Recent simulation studies showed that the MH variances…

The purpose of this study was to investigate the effects of sample size on the power of five selected fit indices through a Monte Carlo simulation. Two models (a reduced and a complete model) and 6 sample sizes (20, 50, 100, 200, 500, and 1,000) were used to investigate the effect on the power of fit indices as the sample size was varied. The…

The probabilities of attaining varying magnitudes of standardized effect sizes by chance and when protected by a 0.05 level statistical test were studied. Monte Carlo procedures were used to generate standardized effect sizes in a one-way analysis of variance situation with 2 through 5, 6, 8, and 10 groups with selected sample sizes from 5 to 500.…

This paper presents results from a comparison of the multiple regression (MR) approach to examining faculty salary equity (with clusters for the various disciplines) and hierarchical linear modeling (HLM) for the same problem. The comparison was done in two steps. First, a practical example of applying both techniques, using empirical data, is…

Test security has often been a problem in computerized adaptive testing (CAT) because the traditional wisdom of item selection overly exposes high discrimination items. The a-stratified (STR) design advocated by H. Chang and his collaborators, which uses items of less discrimination in earlier stages of testing, has been shown to be very…

In the a-stratified method, a popular and efficient item exposure control strategy proposed by H. Chang (H. Chang and Z. Ying, 1999; K. Hau and H. Chang, 2001) for computerized adaptive testing (CAT), the item pool and item selection process has usually been divided into four strata and the corresponding four stages. In a series of simulation…

The purpose of this study was to examine the potential impact of selected methodological factors on the validity of conclusions from reliability generalization (RG) studies. The study focused on four factors; (1) missing data in the primary studies; (2) transformation of sample reliability estimates; (3) use of sample weights for estimating mean…

This paper explores political engagement in an increasingly difficult and troubled world by focusing on how U.S. college students participate in the largest simulation of the United Nations (UN) organization in the United States. The purpose of the paper is to explore what is done in the National Model UN (NMUN) program, its impact on the lives…

In this paper, six statistical tests of stochastic equality are compared with respect to Type I error and power through a Monte Carlo simulation. In the simulation, the skewness and kurtosis levels and the extent of variance heterogeneity of the two parent distributions were varied across a wide range. The sample sizes applied were either small or…

When normality does not hold, nonparametric tests represent an important data-analytic alternative to parametric tests. However, the use of nonparametric tests in educational research has been limited by the absence of easily performed tests for complex experimental designs and analyses, such as factorial designs and multiple regression analyses,…

A study was conducted to investigate whether augmenting the calibration of items using computerized adaptive test (CAT) data matrices produced estimates that were unbiased and improved the stability of existing item parameter estimates. Item parameter estimates from four pools of items constructed for operational use were used in the study to…

Analyses of data from operational course placement systems are subject to the effects of truncation; students with low placement test scores may enroll in a remedial course, rather than a standard-level course, and therefore will not have outcome data from the standard course. In "soft" truncation, some (but not all) students who score…