ERIC Number: EJ1484515
Record Type: Journal
Publication Date: 2025-Sep
Pages: 33
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0276-8739
EISSN: EISSN-1520-6688
Available Date: 2025-06-20
Estimating Heterogeneous Treatment Effects with Item-Level Outcome Data: Insights from Item Response Theory
Joshua B. Gilbert1; Zachary Himmelsbach2; James Soland3; Mridul Joshi4; Benjamin W. Domingue4
Journal of Policy Analysis and Management, v44 n4 p1417-1449 2025
Analyses of heterogeneous treatment effects (HTE) are common in applied causal inference research. However, when outcomes are latent variables assessed via psychometric instruments such as educational tests, standard methods ignore the potential HTE that may exist among the individual items of the outcome measure. Failing to account for "item-level" HTE (IL-HTE) can lead to both underestimated standard errors and identification challenges in the estimation of treatment-by-covariate interaction effects. We demonstrate how Item Response Theory (IRT) models that estimate a treatment effect for each assessment item can both address these challenges and provide new insights into HTE generally. This study articulates the theoretical rationale for the IL-HTE model and demonstrates its practical value using 75 datasets from 48 randomized controlled trials containing 5.8 million item responses in economics, education, and health research. Our results show that the IL-HTE model reveals item-level variation masked by single-number scores, provides more meaningful standard errors in many settings, allows for estimates of the generalizability of causal effects to untested items, resolves identification problems in the estimation of interaction effects, and provides estimates of standardized treatment effect sizes corrected for attenuation due to measurement error.
Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www-wiley-com.bibliotheek.ehb.be/en-us
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305D220046
Department of Education Funded: Yes
Author Affiliations: 1Harvard Graduate School of Education, Cambridge, Massachusetts, USA; 2Massachusetts General Hospital, Center for Addiction Medicine, Boston, Massachusetts, USA; 3University of Virginia School of Education and Human Development, Charlottesville, Virginia, USA; 4Stanford Graduate School of Education, Stanford, California, USA

Peer reviewed
Direct link
