ERIC Number: ED590327
Record Type: Non-Journal
Publication Date: 2018
Pages: 39
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-
EISSN: N/A
Available Date: N/A
Potential for Bias Inflation with Grouped Data: A Comparison of Estimators and a Sensitivity Analysis Strategy
Scott, Marc A.; Diakow, Ronli; Hill, Jennifer L.; Middleton, Joel A.
Grantee Submission, Observational Studies v4 p111-149 2018
We are concerned with the unbiased estimation of a treatment effect in the context of non-experimental studies with grouped or multilevel data. When analyzing such data with this goal, practitioners typically include as many predictors (controls) as possible, in an attempt to satisfy ignorability of the treatment assignment. In the multilevel setting with two levels, there are two classes of potential confounders that one must consider, and attempts to satisfy ignorability conditional on just one set would lead to a different treatment effect estimator than attempts to satisfy the other (or both). The three estimators considered in this paper are so-called "within," "between," and OLS estimators. We generate bounds on the potential differences in bias for these competing estimators to inform model selection. Our approach relies on a parametric model for grouped data and omitted confounders and establishes a framework for sensitivity analysis in the two-level modeling context. The method relies on information obtained from parameters estimated under a variety of multilevel model specifications. We characterize the strength of the confounding and corresponding bias using easily interpretable parameters and graphical displays. We apply this approach to data from a multinational educational evaluation study. We demonstrate the extent to which different treatment effect estimators may be robust to potential unobserved individual- and group-level confounding.
Publication Type: Journal Articles; Reports - Research
Education Level: Secondary Education
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
Identifiers - Location: Italy; United Kingdom (Scotland); Sweden
Identifiers - Assessments and Surveys: International Association for the Evaluation of Educational Achievement Tests
IES Funded: Yes
Grant or Contract Numbers: R305D110037; R305B120017
Author Affiliations: N/A