Peer reviewedERIC Number: ED674603
Record Type: Non-Journal
Publication Date: 2025-Aug-6
Pages: 80
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: 0000-00-00
Quantifying Sensitivity to Selection on Unobserved Covariates: Recasting the Coefficient of Proportionality within a Correlational Framework
Kenneth Frank1; Qinyun Lin2; Spiro Maroulis3; Shimeng Dai, Contributor; Nicole Jess, Contributor; Hung-Chang Lin, Contributor; Yuqing Liu, Contributor; Sarah Maestrales, Contributor; Ellen Searle, Contributor; Jordan Tait, Contributor
Grantee Submission, Paper presented at the Joint Statistical Meetings (Nashville, TN, Aug 6, 2025)
Sensitivity analyses can inform evidence-based education policy by quantifying the hypothetical conditions necessary to change an inference. Perhaps the most prevalent index used for sensitivity analyses is Oster's (2019) Coefficient of Proportionality (COP). Oster's COP leverages changes in estimated effects and R[superscript 2] when observed covariates are added to a model to quantify how strong selection on "unobserved covariates" would have to be relative to on "observed covariates" to nullify an estimated effect. In this paper, we reconceptualize the COP as a function of unobserved covariates' correlations with the focal predictor (e.g., treatment) and with the outcome. Our correlation-based approach addresses recent critiques of Oster's COP while preserving the comparison of selection on unobserved covariates to selection on observed covariates. As importantly, our expressions do not depend on an analyst's subjective choice of covariates to include in a baseline model, reproduce the exact results from Ordinary Least Squares (OLS) estimates even in finite samples, can be adapted to a threshold for inference based on statistical significance, and can be directly calculated from conventionally reported quantities (e.g., estimated effect, standard error) through the Konfound packages in R or Stata. Thus, for most published studies in the social sciences our COP index can be easily applied and intuitively interpreted.
Publication Type: Speeches/Meeting Papers; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305D220022
Department of Education Funded: Yes


