This article presents two alternative methods to null hypothesis significance testing (NHST) for improving inferences from underpowered research designs. Post hoc design analysis (PHDA) assesses whether an NHST analysis generating null findings might otherwise have had sufficient power to detect effects of plausible magnitudes. Bayesian analysis…

Purpose: Null hypothesis significance testing is commonly used in audiology research to determine the presence of an effect. Knowledge of study outcomes, including nonsignificant findings, is important for evidence-based practice. Nonsignificant "p" values obtained from null hypothesis significance testing cannot differentiate between…

Purpose: Evidence-based data analysis methods are important in clinical research fields, including speech-language pathology and audiology. Although commonly used, null hypothesis significance testing (NHST) has several limitations with regard to the conclusions that can be drawn from results, particularly nonsignificant findings. Bayes factors…

Null hypothesis significance testing (NHST) has been the subject of debate for decades and alternative approaches to data analysis have been proposed. This article addresses this debate from the perspective of scientific inquiry and inference. Inference is an inverse problem and application of statistical methods cannot reveal whether effects…

In his article, "An Alternative to Null-Hypothesis Significance Tests," Killeen (2005) urged the discipline to abandon the practice of "p[subscript obs]"-based null hypothesis testing and to quantify the signal-to-noise characteristics of experimental outcomes with replication probabilities. He described the coefficient that he…

This comment offers three descriptions of "p[subscript rep]" that start with a frequentist account of confidence intervals, draw on R. A. Fisher's fiducial argument, and do not make Bayesian assumptions. Links are described among "p[subscript rep]," "p" values, and the probability a confidence interval will capture…

This report presents guidelines for addressing the multiple comparisons problem in impact evaluations in the education area. The problem occurs due to the large number of hypothesis tests that are typically conducted across outcomes and subgroups in these studies, which can lead to spurious statistically significant impact findings. The…

See TM 501 201-3 and EJ 060 883 for related articles. (MLP)

The Behrens-Fisher problem arises when one seeks to make inferences about the means of two normal populations without assuming the variances are equal. This paper presents a review of fundamental concepts and applications used to address the Behrens-Fisher problem under fiducial, Bayesian, and frequentist approaches. Methods of approximations to…

A Bayesian analysis for 2 to the k power factorial arrangements of treatments is presented in this paper. To perform the analysis, an experimenter must specify prior distributions on an orthogonal set of linear functions representing the main effects and interactions and on a function representing the grand mean. The solution is relatively…

See TM 501 202-3 and EJ 060 883 for related articles. (MLP)

A numerical procedure is outlined for obtaining an interval estimate of true score. The procedure is applied to several sets of test data. (Author)

This article compares three models that have been the foundation for approaches to the analysis of statistical significance in early childhood research--the Fisherian and the Neyman-Pearson models (both considered "classical" approaches), and the Bayesian model. The article concludes that all three models have a place in the analysis of research…

The testing of research hypotheses is directly comparable to the dichotomous decision-making of medical diagnosis or jury trials--not ill/ill, or innocent/guilty decisions. There are costs in both kinds of error, type I errors of falsely rejecting a null hypothesis or type II errors of falsely rejecting an alternative hypothesis. It is important…