Factor analysis is widely utilized to identify latent factors underlying the observed variables. This paper presents a comprehensive comparative study of two widely used methods for determining the optimal number of factors in factor analysis, the K1 rule, and parallel analysis, along with a more recently developed method, the bass-ackward method.…

Ensuring a credible literature base is essential for all research fields. One element of this relates to the replicability of published work, which is the probability that the results of an original study would replicate in an independent investigation. A critical feature of replicable research is that the sample size of a study is sufficient to…

In a 1984 essay, Benjamin Bloom, an educational psychologist at the University of Chicago, asserted that tutoring offered "the best learning conditions we can devise" and that tutors could raise student achievement by two full standard deviations--or, in statistical parlance, two "sigmas." The influence of Bloom's two-sigma…

Economic efficiency demands accurate assessment of individual ability for selection purposes. This study investigates Classical Test Theory (CTT) and Item Response Theory (IRT) for estimating true ability and ranking individuals. Two Monte Carlo simulations and real data analyses were conducted. Results suggest a slight advantage for IRT, but…

Multisite cluster randomized trials (MCRTs), in which, the intermediate-level clusters (e.g., classrooms) are randomly assigned to the treatment or control condition within each site (e.g., school), are among the most commonly used experimental designs across a broad range of disciplines. MCRTs often align with the theory that programs are…

There are two schools of thought in statistical analysis, frequentist, and Bayesian. Though the two approaches produce similar estimations and predictions in large-sample studies, their interpretations are different. Bland Altman analysis is a statistical method that is widely used for comparing two methods of measurement. It was originally…

Partially specified correlation matrices (not to be confused with matrices with missing data or EM-correlation matrices) can appear in research settings such as integrative data analyses, quantitative systematic reviews or whenever the study design only allows for the collection of certain variables. Although approaches to fill in these missing…

Analysis of the differential treatment effects across targeted subgroups and contexts is a critical objective in many evaluations because it delineates for whom and under what conditions particular programs, therapies or treatments are effective. Unfortunately, it is unclear how to plan efficient and effective evaluations that include these…

The population relationship between coefficient alpha and scale reliability is studied in the widely used setting of unidimensional multicomponent measuring instruments. It is demonstrated that for any set of component loadings on the common factor, regardless of the extent of their inequality, the discrepancy between alpha and reliability can be…

Imputation or likelihood-based approaches to handle missing data assume the data are missing completely at random (MCAR) or missing at random (MAR). However, little research has examined the missingness pattern before using these imputation/likelihood methods. Three missingness mechanisms -- MCAR, MAR, and not missing at random (NMAR) -- can be…

To date, there have been four responses (Dumas & Edelsbrunner, "Educational Psychology Review," 35, 48, 2023; Grosz, "Educational Psychology Review," 35, 57, 2023; Mayer, "Educational Psychology Review," 35, 64, 2023; Zitzmann et al., "Educational Psychology Review," 35, 65, 2023) to the Brady et al.…

Recruiting participants for studies of early-life longitudinal development is challenging, often resulting in practical upper bounds in sample size and missing data due to attrition. These factors pose risks for the statistical power of such studies depending on the intended analytic model. One mitigation strategy is to increase measurement…

Evidence-based health interventions are frequently translated into real-world settings where practical needs drive changes to intervention protocols. Due to logistical and resource constraints, these naturally arising adaptations are rarely assessed for comparative effectiveness using a randomized trial. Nevertheless, when observational data are…

Several marginal effect size (ES) statistics suitable for quantifying the magnitude of differential item functioning (DIF) have been proposed in the area of item response theory; for instance, the Differential Functioning of Items and Tests (DFIT) statistics, signed and unsigned item difference in the sample statistics (SIDS, UIDS, NSIDS, and…

Meta-analysis of diagnostic test accuracy (DTA) is a powerful statistical method for synthesizing and evaluating the diagnostic capacity of medical tests and has been extensively used by clinical physicians and healthcare decision-makers. However, publication bias (PB) threatens the validity of meta-analysis of DTA. Some statistical methods have…