Quantitative education research is often perceived to be "objective" or "neutral." However, quantitative research has been and continues to be used to perpetuate inequities; these inequities arise as both intended effects and unintended side effects of traditional quantitative research. In this review of the literature, we…

This brief paper introduces a new approach to assessing the trustworthiness of research comparisons when expressed numerically. The 'number needed to disturb' a research finding would be the number of counterfactual values that can be added to the smallest arm of any comparison before the difference or 'effect' size disappears, minus the number of…

The educational literature, the popular press, and educated laypeople have all echoed a conclusion from the book "Academically Adrift" by Richard Arum and Josipa Roksa (which has now become received wisdom), namely, that 45% of college students showed no significant gains in critical thinking skills. Similar results were reported by…

The effective evaluation of correctional programs is critically important. However, research in corrections rarely allows for the randomization of offenders to conditions of the study. This limitation compromises internal validity, and thus, causal conclusions can rarely be drawn. Increasingly, researchers are employing propensity score matching…

This paper proposes a new typology for understanding common research errors that expands on the four types of error commonly discussed in the research literature. Examples are presented to illustrate Type I and Type II errors, errors related to the interpretation of statistically significant and nonsignificant results respectively, with attention…

Although the primary responsibility rests with the authors of articles reporting nonsignificant results to demonstrate the worth of the results by discussing the power of the tests, consumers should be prepared to conduct their own power analyses. This article demonstrates the use of power analysis for the interpretation of nonsignificant…

As an extension of B. Thompson's 1998 invited address to the American Educational Research Association, this paper cites two additional common faux pas in research methodology and explores some research issues for the future. These two errors in methodology are the use of univariate analyses in the presence of multiple outcome variables (with the…

Reports on a review of empirical research reports submitted over the past several years to the "American Journal of Distance Education" that identified common analytic problems and errors often overlooked by distance education researchers. Topics discussed include significance testing; assessment issues, including reliability and…