ERIC Number: ED514524
Record Type: Non-Journal
Publication Date: 2010
Pages: 8
Abstractor: ERIC
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Estimating Long-Term Program Impacts when the Control Group Joins Treatment in the Short-Term: A Theoretical and Empirical Study of the Tradeoffs between Extra- and Quasi-Experimental Estimates
Jaciw, Andrew; Ma, Boya; Zhao, Qingfeng
Society for Research on Educational Effectiveness
Randomized trials of educational interventions often face logistical hurdles (Cook, 2002). For example, sometimes they are prevented from reaching their full-term. This would happen if, as a condition of participation, subjects insist on receiving the treatment program within a certain timeframe. This timeframe may be shorter than the full-term required to get a measure of impact as intended by the program developers. If the controls get exposure to the program before the treatment group gets a full-term exposure, then a valid counterfactual to treatment for measuring the full-term effect is lost, and an alternative has to be established by some other means. In this situation, the researcher has a few options, including: 1) to include an outside comparison group as an alternative control group for when the experimental control group joins treatment (Shadish, Cook and Campbell, 2002); and 2) to estimate impact using an "extra-experimental" method (Bell and Bradley, 2008) which uses impact estimates from earlier birth cohorts in the same experiment to build up a counterfactual for the treatment group. Benefits of the latter approach are that it does not rely on an outside comparison group, and it rules out selection bias and the effects of secular trends that may affect student performance. This work builds on the experimental research tradition in education (Bloom, Bos and Lee, 1999; Boruch et al., 2002; Cook, 2002; Raudenbush, 1997). It can be considered is a first step in investigating the merits and limitation of alternative methods of obtaining program impact estimates in situations where a control group joins treatment before a program has had the opportunity to come full-term. The authors compare the pros and cons of quasi- and extra-experimental impact estimates. The work has both theoretical and empirical components which they outline in this paper.
Descriptors: Control Groups, Research Design, Educational Research, Statistical Analysis, Quasiexperimental Design, Computation, Methods, Reading Instruction, Intervention, Special Education, Program Effectiveness
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries@sree.org; Web site: http://www.sree.org
Publication Type: Reports - Evaluative
Education Level: Elementary Education; Elementary Secondary Education; Grade 3; Grade 4; Grade 5; Grade 6; Grade 7; Grade 8; Intermediate Grades; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A
Author Affiliations: N/A