This letter documents some problems in Ancaiani et al. (2015). Namely the evaluation of concordance, based on Cohen's kappa, reported by Ancaiani et al. was not computed on the whole random sample of 9,199 articles, but on a subset of 7,597 articles. The kappas relative to the whole random sample were in the range 0.07-0.15, indicating an…

The authors of the study "The Long-Term Impact of Teachers" claim that their study shows that increases in teacher value-added lead to significant and lasting increases in test scores and significant increases in income that will last throughout adulthood. Instead, I show that these claims are false because they are contradicted by the…

A dialog between two fictional teachers provides some basic examples of how research that uses approved methodology may provide results that are significant statistically but not significant practically. (PGD)

The second half of a dialogue between two fictional teachers examines the significance of statistical significance in research and considers the factors affecting the extent to which research results provide important or useful information. (PGD)

The dangers of equating "statistical significance" with "real world" significance are summarized. When a finding is said to have "statistical significance," it means only that the same results would be likely to occur again if the study were repeated, not that the finding has any true personal or societal importance. (VW)

Comments on the article by J. S. Hyde (see record 2005-11115-001), in which Hyde reviewed meta-analytic evidence on gender differences and concluded that most psychological gender differences are in the close-to-zero or small range. The current author notes some omissions from Hyde's review, including the findings through other research large…

Problems inherent in relying solely on statistical significance testing as a means of data interpretation are reviewed. The biggest problem with statistical significance testing is that researchers have used the results of this testing to ascribe importance or meaning to their studies where such meaning often does not exist. Often researchers…

In the educational research literature alpha, the a priori level of significance, and p, the a posteriori probability of obtaining a test statistic of at least a certain value when the null hypothesis is true, are often confused. Explanations for this confusion are offered. Paradoxically, alpha retains a prominent place in textbook discussions of…

Hintzman's 1980 attack on certain analyses in memory research is based on doubtful presumptions, namely, that contingency tables are inherently suspect as evidence for or against scientific conclusions and that pressing this logical argument is in some way an acceptable substitute for empirically examining the conclusions in question. (Author)

Argues that a broad dimension of useful experiments would be suggested if the field supplemented its current research strategy with a second strategy focused more explicitly on the extension of generality. (HOD)

Shares three ideas concerning how to evaluate the practical importance of developmental findings to make them more useful to policymakers: (1) statistical significance tests need to be accompanied by effect size estimates; (2) meta-analyses are helping in using all existing data when examining issues involved in policy debates; and (3) researchers…

Three criticisms of overreliance on results from statistical significance tests are noted. It is suggested that: (1) statistical significance tests are often tautological; (2) some uses can involve comparisons that are not completely sensible; and (3) using statistical significance tests to evaluate both methodological assumptions (e.g., the…

Two articles discuss the controversy about the relationship between reliability and the power of significance tests in response to the discussion of Donald W. Zimmerman, Richard H. Williams, and Bruno D. Zumbo. Lloyd G. Humphreys emphasizes the differences between what statisticians can do and constraints on researchers. Zimmerman, Williams, and…

Concerned with imprecision in researchers' use of the word, "interpretive," this report draws from the work of Max Weber to describe the characteristics of an interpretive science of organizational communication and then briefly lists some advantages of following the interpretive approach. First examining the role of subjective meaning…

Presentation of quantitative research on the family often suffers from a tendency to interpret findings on a statistical rather than substantive basis. Advocates the use of data analysis that lends itself to an intuitive understanding of the nature of the findings, the strength of the association, and the import of the result. (JPS)