The paper shows how the methodology for a pedagogical experiment can be improved through including the pre-research stage. If the experiment has the form of a test procedure, an improvement of methodology can be achieved using for example the methods of statistical and didactic analysis of tests which are traditionally used in other areas, i.e.…

The motivation for this paper is the authors' recent work on several randomized control trials in which they found the primary result, which averaged across subgroups or sites, to be moderated by demographic or site characteristics. They are led to examine a distinction that the Institute of Education Sciences (IES) makes between "confirmatory"…

A common mistake in analysis of cluster randomized experiments is to ignore the effect of clustering and analyze the data as if each treatment group were a simple random sample. This typically leads to an overstatement of the precision of results and anticonservative conclusions about precision and statistical significance of treatment effects.…

Very little explanatory power is required in order for regressions to exhibit statistical significance. This article discusses some of the causes and implications. (Contains 2 tables.)

This dissertation was designed to explore the impact of the identified responsibilities of the principal, as an instructional leader, within the professional development model of Reading First. Specifically, the research focused on the principals' use of the Observation and Constructive Feedback Processes as well as five specifically defined…

The authors contend that most research in education lacks statistical power. They feel that the poor use of statistics as a tool for enhancing internal validity must be remediated. The adoption of a new convention is proposed in order to put statistical certainty into reasonable perspective. (RLV)

Three articles in this special issue explore issues related to the use and misuse of statistical significance testing, the major methodological issue in current educational research. Comments on these articles and rejoinders by the authors of the first three articles explore additional aspects of statistical significance testing. (SLD)

Although statistical significance testing as the sole basis for result interpretation is a flawed practice, significance tests can be useful as one of three criteria that must be demonstrated to establish a position empirically. Statistical significance testing provides evidence that an event did not happen by chance but gives no evidence of the…

Expresses a "middle-of-the-road" position on statistical significance testing, suggesting that it has its place but that confidence intervals are generally more useful. Identifies 10 errors of omission or commission in the papers reviewed that weaken the positions taken in their discussions. (SLD)

The use of and emphasis on statistical significance testing has pervaded educational and behavioral research for many decades in spite of criticism by prominent researchers in this field. Much of the controversy is caused by lack of understanding or misinterpretations. This paper reviews criticisms of statistical significance testing and discusses…

Practical significance is concerned with whether a research result is useful in the real world. The use of procedures to supplement the null hypothesis significance test in four journals of the American Psychological Association is examined, and an approach to assessing practical significance is presented. (SLD)

Outlines concerns that must be addressed by those who advocate replacing statistical hypothesis-testing with alternative data-analysis strategies. Suggests that commonly recommended alternatives are not perfect and that various hypothesis-testing modifications can be implemented to make the process and its conclusions more credible. Hypothesis…

Whether one should use null hypothesis testing, confidence intervals, and/or effect sizes is a source of continuing controversy in educational research. An alternative to testing for statistical significance, known as equivalence testing, is little used in educational research. Equivalence testing is useful in situations where the researcher…

The controversy about the use or misuse of statistical significance testing has become the major methodological issue in educational research. This special issue contains three articles that explore the controversy, three commentaries on these articles, an overall response, and three rejoinders by the first three authors. They are: (1)…

Reviews null hypothesis statistical significance testing (NHST) in its historical context and concludes that workable alternatives to NHST are available. Among suggested alternatives, effect magnitude measures, replication techniques, and meta-analytic techniques are discussed. (SLD)