Previous research extensively studied reasons for and ways to avoid low response rates, but it largely ignored the primary research issue of the degree to which response rates matter, which we address. Methodological survey research on response rates has been concerned with how to increase responsiveness and with the effects of response rates on…

This study systematically reviews randomly selected 155 experimental studies in education field originated in the Republic of Turkey between 2010 and 2020. Indiscriminate choice of sample size in recent publications prompted us to evaluate their statistical power and precision. First, above and beyond our review, we could not identify any…

We explored the possibility of publication bias in the sleep and explicit motor sequence learning literature by applying precision effect test (PET) and precision effect test with standard errors (PEESE) weighted regression analyses to the 88 effect sizes from a recent comprehensive literature review (Pan & Rickard, 2015). Basic PET analysis…

Multilevel modeling (MLM) is a statistical technique for analyzing clustered data. Despite its long history, the technique and accompanying computer programs are rapidly evolving. Given the complexity of multilevel models, it is crucial for researchers to provide complete and transparent descriptions of the data, statistical analyses, and results.…

Multivariate analysis of variance (MANOVA) is a statistical method used to examine group differences on multiple outcomes. This article reports results of a review of MANOVA in gifted education journals between 2011 and 2017 (N = 56). Findings suggest a number of conceptual and procedural misunderstandings about the nature of MANOVA and its…

Null hypothesis significance testing (NHST) has been the subject of debate for decades and alternative approaches to data analysis have been proposed. This article addresses this debate from the perspective of scientific inquiry and inference. Inference is an inverse problem and application of statistical methods cannot reveal whether effects…

Multilevel models are an increasingly popular method to analyze data that originate from a clustered or hierarchical structure. To effectively utilize multilevel models, one must have an adequately large number of clusters; otherwise, some model parameters will be estimated with bias. The goals for this paper are to (1) raise awareness of the…

Evaluators of education interventions are increasingly designing studies to detect impacts much smaller than the 0.20 standard deviations that Cohen (1988) characterized as "small." While the need to detect smaller impacts is based on compelling arguments that such impacts are substantively meaningful, the drive to detect smaller impacts…

This "What Works Clearinghouse Procedures and Standards Handbook (Version 3.0)" provides a detailed description of the standards and procedures of the What Works Clearinghouse (WWC). The remaining chapters of this Handbook are organized to take the reader through the basic steps that the WWC uses to develop a review protocol, identify…

Research in fields other than education has found that studies with small sample sizes tend to have larger effect sizes than those with large samples. This article examines the relationship between sample size and effect size in education. It analyzes data from 185 studies of elementary and secondary mathematics programs that met the standards of…

Three criticisms of conventional uses of structural significance testing are elaborated; and alternatives for augmenting statistical significance tests are reviewed, which include emphasizing effect size, evaluating statistical significance in a sample size context, and evaluating result replicability. Among ways of estimating result…

Analysis of 19 studies using omnibus analysis of variance (ANOVA) F-tests resulted in reasons for replacing ANOVA with focused hypothesis testing, which is easy to compile and understand; is flexible; enables trend analysis, determination of confidence intervals, adjustments for violated assumptions, and control of Type 1 errors; makes effect size…

Researchers increasingly recognize that significance tests are limited in their ability to inform scientific practice. Common errors in interpreting significance tests and three strategies for augmenting the interpretation of significance test results are illustrated. The first strategy for augmenting the interpretation of significance tests…

In this paper, tables for the appropriate sample sizes are presented and discussed in the context that the determination of the effect size must precede the determination of the sample size. (Author/PN)

Demonstrates that magnitude-of-effects (ME) estimates vary in susceptibility to sample-size bias depending on whether they are directional or nondirectional estimates. Also demonstrates that study characteristics that influence size of ME estimates can be explicitly taken into account when comparing studies. Emphasizes need to consider study…