The What Works Clearinghouse (WWC) seeks to provide practitioners information about "what works in education." One challenge in understanding "what works" to practitioners is that effect sizes--the degree to which an intervention produces positive (or negative) outcomes--are not comparable across different interventions, in…

This article presents two alternative methods to null hypothesis significance testing (NHST) for improving inferences from underpowered research designs. Post hoc design analysis (PHDA) assesses whether an NHST analysis generating null findings might otherwise have had sufficient power to detect effects of plausible magnitudes. Bayesian analysis…

Purpose: Null hypothesis significance testing is commonly used in audiology research to determine the presence of an effect. Knowledge of study outcomes, including nonsignificant findings, is important for evidence-based practice. Nonsignificant "p" values obtained from null hypothesis significance testing cannot differentiate between…

There is a growing concern that much of the published research literature is distorted by the pursuit of statistically significant results. In a seminal article, Ioannidis and Trikalinos (2007, "Clinical Trials") proposed an omnibus (I&T) test for significance chasing (SC) biases. This test compares the observed number of studies…

Purpose: Evidence-based data analysis methods are important in clinical research fields, including speech-language pathology and audiology. Although commonly used, null hypothesis significance testing (NHST) has several limitations with regard to the conclusions that can be drawn from results, particularly nonsignificant findings. Bayes factors…

Null hypothesis significance testing (NHST) has been the subject of debate for decades and alternative approaches to data analysis have been proposed. This article addresses this debate from the perspective of scientific inquiry and inference. Inference is an inverse problem and application of statistical methods cannot reveal whether effects…

A reproducibility crisis afflicts a wide range of scientific and social-scientific disciplines, from epidemiology to social psychology. Improper research techniques, lack of accountability, disciplinary and political groupthink, and a scientific culture biased toward producing positive results together have produced a critical state of affairs.…