NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED562732
Record Type: Non-Journal
Publication Date: 2014
Pages: 12
Abstractor: ERIC
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Empirical Benchmarks of Hidden Bias in Educational Research: Implication for Assessing How well Propensity Score Methods Approximate Experiments and Conducting Sensitivity Analysis
Dong, Nianbo; Lipsey, Mark
Society for Research on Educational Effectiveness
When randomized control trials (RCT) are not feasible, researchers seek other methods to make causal inference, e.g., propensity score methods. One of the underlined assumptions for the propensity score methods to obtain unbiased treatment effect estimates is the ignorability assumption, that is, conditional on the propensity score, treatment assignment is independent of the outcome. The purpose of this study is to use within-study comparisons to assess how well propensity score methods can approximate experiments under various conditions. The authors test three ways of constructing comparison groups: (1) using the sample from the states that are different from the original experiments, with pretest and demographic information; (2) using the sample from the same state and districts (local matching) with the experiments, with demographic information only; and (3) same as (2) but with pretest as well. Propensity score methods (optimal matching, propensity score weighting, and stratification) are used to estimate the treatment effects for three ways of constructing comparison groups, which are compared with the benchmark from the experiment to assess estimate bias. This study will contribute to the literature by providing empirical evidence about how well propensity score methods can approximate experiments under various conditions. This study uses data from four IES funded projects, among which three are large scaled experiments: (1) "Scaling up TRIAD: Teaching Early Mathematics for Understanding with Trajectories and Technologies"; (2) "Evaluating the Effectiveness of Tennessee's Voluntary Pre-Kindergarten Program"; and (3) "Experimental Evaluation of the Tools of the Mind Pre-K Curriculum", and one is a measurement study, "Learning-Related Cognitive Self-Regulation School Readiness Measures for Preschool Children Study". The analysis is undergoing. The authors report partial results in this report. Tables and figures are appended.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries@sree.org; Web site: http://www.sree.org
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: Massachusetts; New York; North Carolina; Tennessee
Grant or Contract Numbers: N/A
Author Affiliations: N/A