Sparse-rated data are common in operational performance-based language tests, as an inevitable result of assigning examinee responses to a fraction of available raters. The current study investigates the precision of two generalizability-theory methods (i.e., the rating method and the subdividing method) specifically designed to accommodate the…

The future of education is human expertise and artificial intelligence working in conjunction, a revolution that will change the education as we know it. The Intelligent Tutoring System is a key component of this future. A quantitative measurement of efficacies of practice to heterogeneous learners is the cornerstone of building an effective…

A common methodological problem in the evaluation of the predictive validity of selection methods, e.g. in educational and employment selection, is that the correlation between predictor and criterion is biased. Thorndike's (1949) formulas are commonly used to correct for this biased correlation. An alternative approach is to view the selection…

Researchers often collect longitudinal data to model change over time in a phenomenon of interest. Inevitably, there will be some variation across individuals in specific time intervals between assessments. In this simulation study of growth curve modeling, we investigate how ignoring individual differences in time points when modeling change over…

This paper is about performance assessment in serious games. We conceive serious gaming as a process of player-lead decision taking. Starting from combinatorics and item-response theory we provide an analytical model that makes explicit to what extent observed player performances (decisions) are blurred by chance processes (guessing behaviors). We…

One of the central assumptions of the causal-indicator literature is that all causal indicators must be included in the research model and that the exclusion of one or more relevant causal indicators would have severe negative consequences by altering the meaning of the latent variable. In this research we show that the omission of a relevant…

In this article, the authors extend the results of Aguirre-Urreta, Rönkkö, and Marakas (2016) concerning the omission of a relevant causal indicator by testing the validity of the assumption that causal indicators are entirely superfluous to the measurement model and discuss the implications for measurement theory. Contrary to common wisdom…

Correlation attenuation due to measurement error and a corresponding correction, the deattenuated correlation, have been known for over a century. Nevertheless, the deattenuated correlation remains underutilized. A few studies in recent years have investigated factors affecting the deattenuated correlation, and a couple of them provide alternative…

The purpose of this mixed-methods study was to explore Iranian IELTS candidates' strengths and weaknesses in IELTS Speaking Test in terms of IELTS's four speaking assessment criteria, namely Fluency and Coherence (FlC), Lexical Resource (LR), Grammar Range and Accuracy (GRA), and Pronunciation (Pro). It also aimed to examine the discourse features…

Educational analysts studying achievement and other educational outcomes frequently encounter an association between initial status and growth, which has important implications for the analysis of covariate effects, including group differences in growth. As explicated by Allison (1990), where only two time points of data are available, identifying…

A multilevel logistic model for estimating a nonlinear trajectory in a multiple-baseline design is introduced. The model is applied to data from a real multiple-baseline design study to demonstrate interpretation of relevant parameters. A simple change-in-levels (?"Levels") model and a model involving a quadratic function…

To further understand the properties of data-generation algorithms for multivariate, nonnormal data, two Monte Carlo simulation studies comparing the Vale and Maurelli method and the Headrick fifth-order polynomial method were implemented. Combinations of skewness and kurtosis found in four published articles were run and attention was…

Investigating the fit of a parametric model plays a vital role in validating an item response theory (IRT) model. An area that has received little attention is the assessment of multiple IRT models used in a mixed-format test. The present study extends the nonparametric approach, proposed by Douglas and Cohen (2001), to assess model fit of three…

A review of the software Just Another Gibbs Sampler (JAGS) is provided. We cover aspects related to history and development and the elements a user needs to know to get started with the program, including (a) definition of the data, (b) definition of the model, (c) compilation of the model, and (d) initialization of the model. An example using a…

In this paper, we present a simple spreadsheet based simulation activity that can be performed by students at the undergraduate level. This simulation is implemented in free open source software (FOSS) LibreOffice Calc, which is available for both Windows and Linux platform. This activity aims at building the probability distribution for the…