Analyses were conducted to determine whether the two statistical tests of significance are identical. Such demonstrations regarding parallel correlational and analysis of variance procedures have brought further understanding to each of the domains in the past. When a researcher is investigating the relationship between two variables and thinks…

Given some consensus that statistical significance tests are broken, misused, or at least have somewhat limited utility, the focus of discussion within the field ought to move beyond additional bashing of statistical significance tests, and toward more constructive suggestions for improved practice. Five suggestions for improved practice are…

This paper summarizes the literature regarding statistical significance testing with an emphasis on: (1) the post-1994 literature in various disciplines; (2) alternatives to statistical significance testing; and (3) literature exploring why researchers have demonstrably failed to be influenced by the 1994 American Psychological Association…

This paper summarizes the literature regarding statistical significant testing with an emphasis on recent literature in various discipline and literature exploring why researchers have demonstrably failed to be influenced by the American Psychological Association publication manual's encouragement to report effect sizes. Also considered are…

The level of standardized effect sizes obtained by chance and the use of significance tests to guard against spuriously high standardized effect sizes were studied. The concept of the "protected effect size" is also introduced. Monte Carlo methods were used to generate data for the study using random normal deviates as the basis for sample means…

Three articles in this special issue explore issues related to the use and misuse of statistical significance testing, the major methodological issue in current educational research. Comments on these articles and rejoinders by the authors of the first three articles explore additional aspects of statistical significance testing. (SLD)

Although statistical significance testing as the sole basis for result interpretation is a flawed practice, significance tests can be useful as one of three criteria that must be demonstrated to establish a position empirically. Statistical significance testing provides evidence that an event did not happen by chance but gives no evidence of the…

Expresses a "middle-of-the-road" position on statistical significance testing, suggesting that it has its place but that confidence intervals are generally more useful. Identifies 10 errors of omission or commission in the papers reviewed that weaken the positions taken in their discussions. (SLD)

The research methodology literature in recent years has included a full frontal assault on statistical significance testing. An entire edition of "Experimental Education" explored this controversy. The purpose of this paper is to promote the position that while significance testing by itself may be flawed, it has not outlived its…

A national survey of a stratified random sample of members of the American Educational Research Association was undertaken to explore perceptions of contemporary statistical issues, and especially of statistical significance tests. The 225 actual respondents were found to be reasonably representative of the population from which the sample was…

Reviews practices regarding tests of statistical significance and policies of the American Educational Research Association (AERA). Decades of misuse of statistical significance testing are described, and revised editorial policies to improve practice are highlighted. Correct interpretation of statistical tests, interpretation of effect sizes, and…

Three procedures for evaluating the replicability of descriptive discriminant analysis (DDA) results are discussed. The techniques include cross-validation, the jackknife, and the bootstrap. Discriminant analysis is a multivariate technique used when group membership or classification is the focus of the analysis. DDA is used to describe major…

Outlines concerns that must be addressed by those who advocate replacing statistical hypothesis-testing with alternative data-analysis strategies. Suggests that commonly recommended alternatives are not perfect and that various hypothesis-testing modifications can be implemented to make the process and its conclusions more credible. Hypothesis…

Editorial practices revolving around tests of statistical significance are explored. The logic of statistical significance testing is presented in an accessible manner--many people who use statistical tests might not place such a premium on them if they knew what the tests really do, and what they do not do. The etiology of decades of misuse of…

The use of directional and nondirectional hypothesis testing was examined from the perspectives of textbooks, journal articles, and members of editorial boards. Three widely used statistical texts were reviewed in terms of how directional and nondirectional tests of significance were presented. Texts reviewed were written by: (1) D. E. Hinkle, W.…