NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED591605
Record Type: Non-Journal
Publication Date: 2018
Pages: 7
Abstractor: ERIC
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Design-Based Estimators for Average Treatment Effects for Multi-Armed RCTs
Schochet, Peter
Society for Research on Educational Effectiveness
Design-based methods have recently been developed as a way to analyze data from impact evaluations of interventions, programs, and policies (Freedman, 2008; Lin, 2013; Imbens and Rubin, 2015; Schochet, 2013, 2016; Yang and Tsiatis, 2001). The non-parametric estimators are derived using the building blocks of experimental designs with minimal assumptions, and are unbiased and normally distributed in large samples with simple variance estimators. The methods apply to randomized controlled trials (RCTs) and quasi-experimental designs (QEDs) with comparison groups for a wide range of designs used in social policy research. The methods have important advantages over traditional model-based impact estimation methods, such as hierarchical linear model (HLM) and robust cluster standard error (RCSE) methods, and perform well in simulations (Schochet, 2016; Kautz et al., 2017). The literature on design-based methods has focused on RCTs with a single treatment and a single control group. This theory, however, has not been formally extended to designs with multiple research groups. This is an important gap in the literature because multi-armed RCTs can simultaneously examine the effects of multiple interventions in a single study, thereby increasing the amount that researchers and policymakers can learn from impact evaluations. This paper provides new results on the estimation of average treatment effects (ATEs) for multi-armed designs, building on the design-based literature for the two-group design. The approach is based on the Neyman-Rubin-Holland potential outcomes framework that underlies experiments (Holland, 1986; Neyman, 1923, Rubin, 1974, 1977). The paper considers both non-clustered and clustered designs as well as designs with blocking and baseline covariates. The paper also discusses how design-based ATE estimators for the two-group design need to be modified for the multi-armed design when comparing pairs of research groups to each other. This paper presents an empirical example using data from a multi-armed RCT testing the effects of various supplemental reading interventions. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A
Author Affiliations: N/A