Application of MIRT modeling procedures is dependent on the quality of parameter estimates provided by the estimation software and techniques used. This study investigated model parameter recovery of two popular MIRT packages, BMIRT and flexMIRT, under some common measurement conditions. These packages were specifically selected to investigate the…

When undertaking quantitative hypothesis testing, social researchers need to decide whether the data with which they are working is suitable for parametric analyses to be used. When considering the relevant assumptions they can examine graphs and summary statistics but the decision making process is subjective and must also take into account the…

When researchers fit statistical models to multiply imputed datasets, they have to fit the model separately for each imputed dataset. Since there are multiple datasets, there are always multiple sets of model results. It is possible for some of these sets of results not to converge while some do converge. This study examined occurrence of such a…

In education, sample characteristics can be complex due to the nested structure of students, teachers, classrooms, schools, and districts. In the past, not many considerations were given to such complex sampling schemes in statistical power analysis. More recently in the past two decades, however, education scholars have developed tools to conduct…

The What Works Clearinghouse (WWC) is an initiative of the U.S. Department of Education's Institute of Education Sciences (IES), which was established under the Education Sciences Reform Act of 2002. It is an important part of IES's strategy to use rigorous and relevant research, evaluation, and statistics to improve the nation's education system.…

Automated writing evaluation (AWE) is a cutting-edge technology-based intervention designed to help teachers meet their challenges in writing classrooms and improve students' writing proficiency. The fast development of AWE systems, along with the encouragement of technology use in the U.S. K-12 education system by the Common Core State Standards…

Although content validity--how well each item of an instrument represents the construct being measured--is foundational in the development of an instrument, statistical validity is also important to the decisions that are made based on the instrument. The primary purpose of this study is to demonstrate how simulation studies can be used to assist…

Bayesian algorithms have been used successfully in the social and behavioral sciences to analyze dichotomous data particularly with complex structural equation models. In this study, we investigate the use of the Polya-Gamma data augmentation method with Gibbs sampling to improve estimation of structural equation models with dichotomous variables.…

The uncertainty arising from item parameter estimation is often not negligible and must be accounted for when calculating latent variable (LV) scores in item response theory (IRT). It is particularly so when the calibration sample size is limited and/or the calibration IRT model is complex. In the current work, we treat two-stage IRT scoring as a…

This is the technical appendix for the report "Gaining Momentum: Refining Corequisite Learning Support to Boost Student Success in the First Year and Beyond" which seeks to develop a body of actionable information about how to refine learning support, close equity gaps, and promote strong momentum toward success for community college…

Coefficient omega and alpha are both measures of the composite reliability for a set of items. Unlike coefficient alpha, coefficient omega remains unbiased with congeneric items with uncorrelated errors. Despite this ability, coefficient omega is not as widely used and cited in the literature as coefficient alpha. Reasons for coefficient omega's…

An important concern when planning research studies is to obtain maximum precision of an estimate of a treatment effect given a budget constraint. When research designs have a "multilevel" or "hierarchical" structure changes in sample size at different levels of the design will impact precision differently. Furthermore, there…

In item response theory (IRT), the invariance property states that item parameter estimates are independent of the examinee sample, and examinee ability estimates are independent of the test items. While this property has long been established and understood by the measurement community for IRT models, the same cannot be said for diagnostic…

Two new methods for the meta-analysis of factor loadings are introduced and evaluated by Monte Carlo simulations. The direct method pools each factor loading individually, whereas the indirect method synthesizes correlation matrices reproduced from factor loadings. The results of the two simulations demonstrated that the accuracy of…

The value of a study relies heavily on the quality of data collection and analysis. No matter the skill level of the researchers or what journal the study is published in, poor collection and management of data result in a poor study. Our personal experiences and impressions of collecting, storing, protecting, and retrieving data for a U.S.…