Many experimental designs in educational and behavioral research involve at least one level of clustering. Clustering affects the precision of estimators and its impact on statistics in cross-sectional studies is well known. Clustering also occurs in longitudinal designs where students that are initially grouped may be regrouped in the following…

Students often believe that statistical significance is the only determinant of whether a quantitative result is "important." In this paper, I review traditional null hypothesis statistical testing to identify what questions inferential statistics can and cannot answer, including statistical significance, effect size and direction,…

Most education reporters from time to time will tread into the world of education research, whether to gauge charter school achievement, the impact of teacher quality, or the effects of a reading program, among myriad possibilities. But making sense of the research, with its often-impenetrable prose, dizzying figures, and mathematical formulas,…

The self-correcting nature of psychological and educational science has been seriously questioned. Recent special issues of "Perspectives on Psychological Science" and "Psychology of Aesthetics, Creativity, and the Arts" have roundly condemned current organizational models of research and dissemination and have criticized the…

The identification of evidence-based practices continues to provoke issues of disagreement across multiple fields. One area of contention is the role of single-subject design (SSD) research in providing scientific evidence. The debate about SSD's utility centers on three issues: sample size, effect size, and serial dependence. One potential…

With growing popularity of single subject design as a method to evaluate the efficacy of interventions, it is important to ensure that the analyses of these methods are rigorous and reliable. The purpose of this case study is to discuss the measures used to evaluate the efficacy of interventions in single subject design studies in the fields of…

In some instances, intentionally or not, study designs are such that there is clustering in one group but not in the other. This paper describes methods for computing effect size estimates and their variances when there is clustering in only one group and the analysis has not taken that clustering into account. The authors provide the effect size…

This meta-analysis synthesizes the literature on interventions for struggling readers in Grades 4 through 12 published between 1980 and 2011. It updates Scammacca et al.'s analysis of studies published between 1980 and 2004. The combined corpus of 82 study-wise effect sizes was meta-analyzed to determine (a) the overall effectiveness of reading…

Purpose: The purpose of this study was to examine advanced coursework completion rates, Advanced Placement (AP)/International Baccalaureate (IB) testing rates, AP/IB exam passage rates, and the percentage of AP/IB exam scores at or above the criterion that may exist among Texas public high school students from 2001 to 2012 to ascertain (a) the…

This methodological brief introduces the readers to the propensity score matching method, which can be used for enhancing the validity of causal inferences in research situations involving nonexperimental design or observational research, or in situations where the benefits of an experimental design are not fully realized because of reasons beyond…

The sense that replicability is an important aspect of empirical science led Killeen (2005a) to define "p[subscript rep]," the probability that a replication will result in an outcome in the same direction as that found in a current experiment. Since then, several authors have praised and criticized 'p[subscript rep]," culminating…

Previous work on statistical power has discussed mainly single-level designs or 2-level balanced designs with random effects. Although balanced experiments are common, in practice balance cannot always be achieved. Work on class size is one example of unbalanced designs. This study provides methods for power analysis in 2-level unbalanced designs…

A pre-post experimental design with 6-week delayed posttest was implemented to investigate the effects of reading comprehension strategy (RCS) instruction with and without attribution retraining (AR) on reading outcomes for seventh, eighth, and ninth graders with learning and other mild disabilities. Students were randomly assigned to one of three…

The increased interest in reporting effect sizes means that it is necessary to consider what should be included in a primer on effect sizes. A review of papers on effect sizes and commonly repeated statistical analyses suggests that it is important to discuss effect sizes relative to bivariate correlation, t-tests, analysis of variance/covariance,…

Controversy over the proper place of significance testing within scientific methodology has continued for some time. The suggestion that effect sizes are more important than whether results are significant is presented. Effect size can be defined as an estimate of how much of the dependent variable is accounted for by the independent variables.…