Research designs that include planned missing data are gaining popularity in applied education research. These methods have traditionally relied on introducing missingness into data collections using the missing completely at random (MCAR) mechanism. This study assesses whether planned missingness can also be implemented when data are instead…

Meta-analysis is a statistical method that is increasingly utilized to combine and compare the results of previous primary studies. However, because of the lack of comprehensive guidelines for how to use meta-analysis, many meta-analysis studies have failed to consider important aspects, such as statistical programs, power analysis, publication…

Research in fields other than education has found that studies with small sample sizes tend to have larger effect sizes than those with large samples. This article examines the relationship between sample size and effect size in education. It analyzes data from 185 studies of elementary and secondary mathematics programs that met the standards of…

Secondary data analysis as a methodological approach is not without its critics. Indeed, three main objections to the use of secondary data analysis in social research stand out: first that because of the socially constructed nature of social data, the act of reducing it to a simple numeric form cannot fully encapsulate its complexity. Secondly,…

This article presents the author's address at the 2007 "Journal of Applied Quantitative Methods" ("JAQM") prize awarding festivity. The festivity was included in the opening of the 4th International Conference on Applied Statistics, November 22, 2008, Bucharest, Romania. In the address, the author reflects on three theses that…

Classic parametric statistical significance tests, such as analysis of variance and least squares regression, are widely used by researchers in many disciplines, including psychology. For classic parametric tests to produce accurate results, the assumptions underlying them (e.g., normality and homoscedasticity) must be satisfied. These assumptions…

Compared two types of empirical study with two types of conceptual research on the validity of the Rorschach. Argues that the Rorschach does have some validity and that poor research is at least partly culpable for the Rorschach's perceived failure. (Author/BL)

This article looks at the methodological problems created by incorrectly using between-class data when looking at effects generated by individual students. The grade/evaluation association found in the student evaluation of teaching was used as a case example. A theoretical framework to understand the contributions to the relationship by both…

Compelling arguments for reform of statistical practices have been made in many disciplines, in some cases over several decades, but achieving reform has proved difficult. We discuss how reform has progressed--or not progressed--in psychology, medicine, and ecology and describe case studies of attempts by pioneering journal editors to change…

In the educational research literature alpha, the a priori level of significance, and p, the a posteriori probability of obtaining a test statistic of at least a certain value when the null hypothesis is true, are often confused. Explanations for this confusion are offered. Paradoxically, alpha retains a prominent place in textbook discussions of…

Calls attention to limitations and dangers in routine application of multivariate analysis of variance (MANOVA), describes some alternative procedures, and laments the necessarily pervasive character of the statistical problem. (Author)

Presents several multivariate analyses of variance (MANOVA) test procedures. Discusses guidelines for choosing an overall MANOVA test statistic and post hoc tests that determine the dependent variable or variables responsible for any significant effects. Concludes that guidelines based on recent comparisons of the various test statistics be used.…

Meta-analysis has become an important supplement to traditional methods of research reviewing, although many problems must be addressed by the reviewer who carries out a meta-analysis. These problems include identifying and obtaining appropriate studies, extracting estimates of effect size from the studies, coding or classifying studies, analyzing…

Methods for selecting relevant and appropriate quasi-experimental studies for inclusion in research synthesis using the threats-to-validity approach are presented. Effects of including and excluding studies are evaluated. (BS)

Major changes in the methods and contents of research are needed. Promising trends in the design and conduct of studies include: (a) macroscopic rather than microscopic levels of study; (b) field rather than laboratory studies; (c) focusing on the total individual; (d) developing a contractual relationship between researcher and subject. (Author)