ERIC Number: EJ1305519
Record Type: Journal
Publication Date: 2021
Pages: 23
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0276-8739
EISSN: N/A
Available Date: N/A
Lurking Inferential Monsters? Quantifying Selection Bias in Evaluations of School Programs
Weidmann, Ben; Miratrix, Luke
Journal of Policy Analysis and Management, v40 n3 p964-986 Sum 2021
This study examines whether unobserved factors substantially bias education evaluations that rely on the Conditional Independence Assumption. We add 14 new within-study comparisons to the literature, all from primary schools in England. Across these 14 studies, we generate 42 estimates of selection bias using a simple approach to observational analysis. A meta-analysis of these estimates suggests that the distribution of underlying bias is centered around zero. The mean absolute value of estimated bias is 0.03s, and none of the 42 estimates are larger than 0.11s. Results are similar for math, reading, and writing outcomes. Overall, we find no evidence of substantial selection bias due to unobserved characteristics. These findings may not generalize easily to other settings or to more radical educational interventions, but they do suggest that non-experimental approaches could play a greater role than they currently do in generating reliable causal evidence for school education.
Descriptors: Foreign Countries, Program Evaluation, Bias, Elementary Schools, Selection, Intervention
Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www-wiley-com.bibliotheek.ehb.be/en-us
Publication Type: Journal Articles; Information Analyses
Education Level: Elementary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: United Kingdom (England)
Grant or Contract Numbers: N/A
Author Affiliations: N/A