In this study, we re-analyze recent empirical research on replication from a meta-analytic perspective. We argue that there are different ways to define "replication failure," and that analyses can focus on exploring variation among replication studies or assess whether their results contradict the findings of the original study. We…

Classic parametric statistical significance tests, such as analysis of variance and least squares regression, are widely used by researchers in many disciplines, including psychology. For classic parametric tests to produce accurate results, the assumptions underlying them (e.g., normality and homoscedasticity) must be satisfied. These assumptions…

Surveyed AERA members regarding their perceptions of: statistical issues and statistical significance testing; the general linear model; stepwise methods; score reliability; type I and II errors; sample size; statistical probabilities as exclusive measures of effect size; p values as direct measures of result value; and p values evaluating…

After presenting a general linear model as a framework for discussion, this paper reviews five methodology errors that occur in educational research: (1) the use of stepwise methods; (2) the failure to consider in result interpretation the context specificity of analytic weights (e.g., regression beta weights, factor pattern coefficients,…

Some of the methods originally used to find relationships between reliability and power associated with a single measurement are extended to difference scores. Results, based on explicit power calculations, show that augmenting the reliability of measurement by reducing error score variance can make significance tests of difference more powerful.…

Two articles discuss the controversy about the relationship between reliability and the power of significance tests in response to the discussion of Donald W. Zimmerman, Richard H. Williams, and Bruno D. Zumbo. Lloyd G. Humphreys emphasizes the differences between what statisticians can do and constraints on researchers. Zimmerman, Williams, and…

Dissertations are an important component of the effort to generate knowledge. Thus, dissertation quality may be seen by accreditation and coordinating-board reviewers as a noteworthy reflection on the quality of doctoral programs themselves. The present study reviews methodological errors within Ph.D. dissertations. The illustrative errors are…