The internal validity of observational study is often subject to debate. In this study, we define the counterfactuals as the unobserved sample and intend to quantify its relationship with the null hypothesis statistical testing (NHST). We propose the probability of a robust inference for internal validity, that is, the PIV, as a robustness index…

A Monte Carlo simulation study was conducted to compare fit indices used for detecting the correct latent class in three dichotomous mixture item response theory (IRT) models. Ten indices were considered: Akaike's information criterion (AIC), the corrected AIC (AICc), Bayesian information criterion (BIC), consistent AIC (CAIC), Draper's…

Publication bias threatens meta-analysis validity. It is often assessed via the funnel plot; an asymmetric plot implies small-study effects, and publication bias is one cause of the asymmetry. Egger's regression test is a widely used tool to quantitatively assess such asymmetry. It examines the association between the observed effect sizes and…

This article presents two alternative methods to null hypothesis significance testing (NHST) for improving inferences from underpowered research designs. Post hoc design analysis (PHDA) assesses whether an NHST analysis generating null findings might otherwise have had sufficient power to detect effects of plausible magnitudes. Bayesian analysis…

Although there is recognition that there may be differential outcomes for groups of students within schools, examination of outcomes for subgroups presents challenges to researchers and policymakers. It complicates analytic procedures, particularly when the number of students per school in the subgroup is small. We explored five alternatives for…

Purpose and Background: Despite the flexibility of multilevel structural equation modeling (MLSEM), a practical limitation many researchers encounter is how to effectively estimate model parameters with typical sample sizes when there are many levels of (potentially disparate) nesting. We develop a method-of-moment corrected maximum likelihood…

Bayesian structural equation modeling (BSEM) is a flexible tool for the exploration and estimation of sparse factor loading structures; that is, most cross-loading entries are zero and only a few important cross-loadings are nonzero. The current investigation was focused on the BSEM with small-variance normal distribution priors (BSEM-N) for both…

This study evaluated the validity of the Michigan English Test (MET) Listening Section by investigating its underlying factor structure and the replicability of its factor structure across multiple test forms. Data from 3255 test takers across four forms of the MET Listening Section were used. To investigate the factor structure, each form was…

Although extensive research has focused on the perceptual abilities of second language (L2) learners, a significant gap persists in understanding how cognitive functions like phonological short-term memory (PSTM) and nonverbal intelligence (IQ) impact L2 speech perception. This study sets out to investigate the discrimination of L2 English…

This study explored the use of stealth assessments within a digital game to assess second language (L2) Chinese learners' reading comprehension. Log data tracking learners' in-game behaviors from a game designed for Chinese dual language immersion classrooms (Poole et al., 2022) were used to construct Bayesian Belief Networks to model reading…

This paper explores how student performance across the full set of multiple modular assessments of individual standards, which we refer to as mini-assessments, from a large scale, operational program of interim assessment can be summarized using Bayesian networks. We follow a completely data-driven approach in which no constraints are imposed to…

Although some previous studies have investigated the relationship between moral foundations and moral judgment development, the methods used have not been able to fully explore the relationship. In the present study, we used Bayesian Model Averaging (BMA) in order to address the limitations in traditional regression methods that have been used…

Despite decades of research, it has been difficult to achieve consensus on a definition of common learning disabilities such as dyslexia. This lack of consensus represents a fundamental problem for the field. Our approach to addressing this issue is to use model-based meta-analyses and Bayesian models with informative priors to combine the results…

Synthesizing findings about the indirect (mediated) effect plays an important role in determining the mechanism through which variables affect one another. This simulation study compared six methods for synthesizing indirect effects: correlation-based MASEM, parameter-based MASEM, marginal likelihood synthesis, an adjustment to marginal likelihood…

This study aims to create learning path navigation for target learners by discovering the correlation among micro-learning units. In this study, the learning path is defined as a sequence of learning units used to realize a learning goal, and a period used for realizing the learning goal is regarded as a learning cycle. Furthermore, the learning…