Does "X" affect "Y"? Answering this question is particularly difficult if reverse causality is looming. Many social scientists turn to panel data to address such questions of causal ordering. Yet even in longitudinal analyses, reverse causality threatens causal inference based on conventional panel models. Whereas the…

Randomised trials are often funded by commercial companies and methodological studies support a widely held suspicion that commercial funding may influence trial results and conclusions. However, these studies often have a risk of confounding and reporting bias. The risk of confounding is markedly reduced in meta-epidemiological studies that…

Randomized clinical trials underpin evidence-based clinical practice, but flaws in their conduct may lead to biased estimates of intervention effects and hence invalid treatment recommendations. The main approach to the empirical study of bias is to collate a number of meta-analyses and, within each, compare the results of trials with and without…

Treatment effect estimates from a "regression discontinuity design" (RDD) have high internal validity. However, the arguments that support the design apply to a subpopulation that is narrower and usually different from the population of substantive interest in evaluation research. The disconnect between RDD population and the…

Regression discontinuity design (RD) has been widely used to produce reliable causal estimates. Researchers have validated the accuracy of RD design using within study comparisons (Cook, Shadish & Wong, 2008; Cook & Steiner, 2010; Shadish et al, 2011). Within study comparisons examines the validity of a quasi-experiment by comparing its…

Randomized controlled trials (RCTs) and regression discontinuity (RD) studies both provide estimates of causal effects. A major difference between the two is that RD only estimates local average treatment effects (LATE) near the cutoff point of the forcing variable. This has been cited as a drawback to RD designs (Cook & Wong, 2008).…

This study compared college student reported grade point averages (GPA) with actual GPA as recorded at the Registrar's Office to determine the accuracy of student reported GPA. Results indicated that, on average, students reported slightly higher GPA than their actual GPA. Additionally, females were virtually as accurate as males and students with…

Schools and school systems throughout the nation are increasingly experimenting with using various instructional technologies to improve productivity and decrease costs, but evidence on both the effectiveness and the costs of education technology is limited. A recent report published by the Thomas B. Fordham Institute sets out to describe "the…

The Center for Research on Education Outcomes (CREDO) at Stanford University conducted a large-scale analysis of the impact of charter schools on student performance. The center's data covered 65-70% of the nation's charter schools. Although results varied by state, 17% of the charter school students have significantly higher math results than …

"How New York City's Charter Schools Affect Achievement" estimates the effects on student achievement of attending a New York City charter school rather than a traditional public school and investigates the characteristics of charter schools associated with the most positive effects on achievement. Because the report relies on an…

Evaluations involving nonrandom assignment to treatment or control groups are vulnerable to an accidental or intentional confounding of a selection effect with the treatment effect. Two techniques, discriminant analysis and base expectancy analysis, permit separate estimation of the selection and treatment effects in the final results. (Author/CTM)

Ignorance of the characteristics of a mixed population may lead to bias in a summary measure of a phenomenon. A test based on sample kurtosis is demonstrated by a simulation study to be more powerful than six other known tests in detecting a class of mixed normal distributions. (SLD)

The document, part of a series of chapters described in SO 011 759, considers the problem of censoring in the analysis of event-histories (data on dated events, including dates of change from one qualitative state to another). Censoring refers to the lack of information on events that occur before or after the period for which data are available.…

Abuses and misuses of statistics are frequent. This digest attempts to warn against these in three broad classes of pitfalls: sources of bias, errors of methodology, and misinterpretation of results. Sources of bias are conditions or circumstances that affect the external validity of statistical results. In order for a researcher to make…

The conclusion of a 1999 Organisation for Economic Cooperation and Development (OECD) report that wage gains for training are higher for workers with lower levels of education was revisited using data for males from the 1997 Australian Survey of Education and Training (SET). The study used methods similar to the OECD report (ordinary least squares…