The What Works Clearinghouse (WWC) seeks to provide practitioners information about "what works in education." One challenge in understanding "what works" to practitioners is that effect sizes--the degree to which an intervention produces positive (or negative) outcomes--are not comparable across different interventions, in…

This article presents two alternative methods to null hypothesis significance testing (NHST) for improving inferences from underpowered research designs. Post hoc design analysis (PHDA) assesses whether an NHST analysis generating null findings might otherwise have had sufficient power to detect effects of plausible magnitudes. Bayesian analysis…

Purpose: Null hypothesis significance testing is commonly used in audiology research to determine the presence of an effect. Knowledge of study outcomes, including nonsignificant findings, is important for evidence-based practice. Nonsignificant "p" values obtained from null hypothesis significance testing cannot differentiate between…

There is a growing concern that much of the published research literature is distorted by the pursuit of statistically significant results. In a seminal article, Ioannidis and Trikalinos (2007, "Clinical Trials") proposed an omnibus (I&T) test for significance chasing (SC) biases. This test compares the observed number of studies…

Purpose: Evidence-based data analysis methods are important in clinical research fields, including speech-language pathology and audiology. Although commonly used, null hypothesis significance testing (NHST) has several limitations with regard to the conclusions that can be drawn from results, particularly nonsignificant findings. Bayes factors…

Null hypothesis significance testing (NHST) has been the subject of debate for decades and alternative approaches to data analysis have been proposed. This article addresses this debate from the perspective of scientific inquiry and inference. Inference is an inverse problem and application of statistical methods cannot reveal whether effects…

A reproducibility crisis afflicts a wide range of scientific and social-scientific disciplines, from epidemiology to social psychology. Improper research techniques, lack of accountability, disciplinary and political groupthink, and a scientific culture biased toward producing positive results together have produced a critical state of affairs.…

Person-fit assessment may help the researcher to obtain additional information regarding the answering behavior of persons. Although several researchers examined person fit, there is a lack of research on person-fit assessment for mixed-format tests. In this article, the lz statistic and the ?2 statistic, both of which have been used for tests…

Most biological scientists conduct experiments to look for effects, and test the results statistically. One of the commonly used test is Student's t test. However, this test concentrates on a very limited question. The authors assume that there is no effect in the experiment, and then estimate the possibility that they could have obtained these…

Applied researchers often find themselves making statistical inferences in settings that would seem to require multiple comparisons adjustments. We challenge the Type I error paradigm that underlies these corrections. Moreover we posit that the problem of multiple comparisons can disappear entirely when viewed from a hierarchical Bayesian…

The objectives of this report are: (a) to trace the theoretical roots of the concept clinical significance that derives from Bayesian thinking, Marginal Utility/Diminishing Returns in Economics, and the "just noticeable difference", in Psychophysics. These concepts then translated into: Effect Size (ES), strength of agreement, clinical…

This meta-analysis synthesizes the literature on interventions for struggling readers in Grades 4 through 12 published between 1980 and 2011. It updates Scammacca et al.'s analysis of studies published between 1980 and 2004. The combined corpus of 82 study-wise effect sizes was meta-analyzed to determine (a) the overall effectiveness of reading…

Information technology adoption and diffusion is currently a significant challenge in the healthcare delivery setting. This thesis includes three papers that explore social influence on information technology adoption and sustained use in the healthcare delivery environment using conventional regression models and novel hierarchical Bayesian…

During the spring of 2012, over 100,000 educators across the state took the North Carolina Teacher Working Conditions (TWC) survey. Wake County Public School System (WCPSS) teachers responded more positively to 46 of the 85 survey items in 2012 than did teachers statewide. On 14 items, WCPSS teachers' responses were less favorable than those of…

KIPP (Knowledge is Power Program) is a national network of public charter schools whose stated mission is to help underserved students enroll in and graduate from college. Prior studies (see Tuttle et al. 2013) have consistently found that attending a KIPP middle school positively affects student achievement, but few have addressed longer-term…