Sensitivity analyses can inform evidence-based education policy by quantifying the hypothetical conditions necessary to change an inference. Perhaps the most prevalent index used for sensitivity analyses is Oster's (2019) Coefficient of Proportionality (COP). Oster's COP leverages changes in estimated effects and R[superscript 2] when observed…

The purpose of this paper is to elaborate and comment on some themes related to construct validity, construct choice, and construct respecification in the Campbellian validity system. It is argued that Reichardt's [(2008, November). An alternative to the Campbellian conceptualization of validity. (Paper presentation). Evaluation 2008 Conference,…

Pearson's correlation is widely used to test for an association between two variables and also forms the basis of several multivariate statistical procedures including many latent variable models. Spearman's [rho] is a popular alternative. These procedures are compared with ranking the data and then applying the inverse normal transformation, or…

Data pooling is a powerful strategy in empirical research. However, combining multiple datasets often results in a large amount of missing data, as variables that are not present in some datasets effectively contain missing values for all participants in those datasets. Furthermore, data pooling typically leads to a mix of continuous and…

Causal mediation analysis has gained increasing attention in recent years. This article guides empirical researchers through the concepts and challenges of causal mediation analysis. I first clarify the difference between traditional and causal mediation analysis and highlight the importance of adjusting for the treatment-by-mediator interaction…

School start regulations allocate children born immediately either side of a given date to different life paths: those slightly older starting school a full year earlier. School effectiveness literature exploits this to estimate causal effects described as 'the absolute effect of schooling' or 'the effect of an additional year's schooling', using…

Over the past decade there has been a striking increase in the number of quantitative studies examining the effects of social mobility, with almost all based on the diagonal reference model (DRM). We make four main contributions to this rapidly expanding literature. First, we show that under plausible values of mobility effects, the DRM will, in…

The variance-stabilizing Freeman-Tukey double arcsine transform was originally proposed for inference on single proportions. Subsequently, its use has been suggested in the context of meta-analysis of proportions. While some erratic behavior has been observed previously, here we point out and illustrate general issues of monotonicity and…

This article argues that writing studies should perform something it terms "construct-focused RAD research," or quantitative RAD research involving psychometric measurements, to study cognitive constructs that pertain to student writing. Construct-focused RAD research, which today is rare in the field, can provide a clearer sense of what…

Test takers wishing to gain an unfair advantage often share answers with other test takers, either sharing all answers (a full key) or some (a partial key). Detecting key sharing during a tight testing window requires an efficient, easily interpretable, and rich form of analysis that is descriptive and inferential. We introduce a detection method…

The import or force of the result of a statistical test has long been portrayed as consistent with deductive reasoning. The simplest form of deductive argument has a first premise with conditional form, such as p[right arrow]q, which means that "if p is true, then q must be true." Given the first premise, one can either affirm or deny…

The multi-decade debates within the field of behavior analysis as to the possible value and threat of group design methodology and statistical analyses on the purity of the field have weakened the discipline's maximal impact on the world. This paper rebukes the concerns and suggests that through such adoption behavior analysis may likely better…

Given recent evidence challenging the replicability of results in the social and behavioral sciences, critical questions have been raised about appropriate measures for determining replication success in comparing effect estimates across studies. At issue is the fact that conclusions about replication success often depend on the measure used for…

Education research has recently seen the emergence of two distinct frameworks guiding the application of quantitative methods through a more critical and equity-oriented lens. These two frameworks are critical quantitative (CritQuant) studies and quantitative critical race theory (QuantCrit). Although different in their intellectual traditions,…

Ensuring a credible literature base is essential for all research fields. One element of this relates to the replicability of published work, which is the probability that the results of an original study would replicate in an independent investigation. A critical feature of replicable research is that the sample size of a study is sufficient to…