NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED622050
Record Type: Non-Journal
Publication Date: 2022
Pages: 45
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Covariate Balance for Observational Effectiveness Studies: A Comparison of Matching and Weighting
Joseph M. Kush; Elise T. Pas; Rashelle J. Musci; Catherine P. Bradshaw
Grantee Submission
Propensity score matching and weighting methods are often used in observational effectiveness studies to reduce imbalance between treated and untreated groups on a set of potential confounders. However, much of the prior methodological literature on matching and weighting has yet to examine performance for scenarios with a majority of treated units, as is often encountered with programs and interventions that have been widely disseminated or "scaled up". Using a series of Monte Carlo simulations, we compare the performance of k:1 matching with replacement and weighting methods with respect to covariate balance, bias, and MSE. Results indicate that the accuracy of all methods declined as treatment prevalence increased. While weighting produced the largest reduction in covariate imbalance, 1:1 matching with replacement provided the most unbiased treatment effect estimates. An applied example using empirical school-level data is provided to further illustrate the application and interpretation of these methods to a real-world scale-up effort. We conclude by considering the implications of propensity score methods for observational effectiveness studies with a particular focus on educational research. [This paper will be published in "Journal of Research on Educational Effectiveness."]
Related Records: EJ1387554
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305H150027; R305A150221
Author Affiliations: N/A