Randomized controlled trials have proliferated in education, in part because they provide an unbiased estimator for the causal impact of interventions. It is increasingly recognized that many such trials in education have low power to detect an effect if indeed there is one. However, it is less well known that low powered trials tend to…

Background: Over the past decade, research on the development and assessment of tools to improve the generalizability of experimental findings has grown extensively (Tipton & Olsen, 2018). However, many experimental studies in education are based on small samples, which may include 30-70 schools while inference populations to which…

Omitted variable bias occurs in most statistical models. Whenever a confounding variable that is correlated with both dependent and independent variables is omitted from a statistical model, estimated effects of included variables are likely to be biased due to omitted variables. This issue is particularly problematic in physics education research…

Evaluators of education interventions are increasingly designing studies to detect impacts much smaller than the 0.20 standard deviations that Cohen (1988) characterized as "small." While the need to detect smaller impacts is based on compelling arguments that such impacts are substantively meaningful, the drive to detect smaller impacts…

Multilevel models are often used to evaluate hypotheses about relations among constructs when data are nested within clusters (Raudenbush & Bryk, 2002), although alternative approaches are available when analyzing nested data (Binder & Roberts, 2003; Sterba, 2009). The overarching goal of this article is to suggest when it is appropriate…

Online surveys are increasingly used in educational research, yet little attention has focused on ethical issues associated with their use in educational settings. Here, we draw on the broader literature to discuss 5 key ethical issues in the context of educational survey research: dual teacher/researcher roles; informed consent; use of…

Educational researchers commonly use the rule of thumb of "design effect smaller than 2" as the justification of not accounting for the multilevel or clustered structure in their data. The rule, however, has not yet been systematically studied in previous research. In the present study, we generated data from three different models…

Valid, scientific research is critical for ascertaining the effects of instructional techniques on learners with disabilities and for guiding effective special education practice and policy. Researchers in fields such as psychology and medicine have identified serious and widespread shortcomings in their research literatures related to replication…

This article contains a second-order meta-analysis and an exploration of bias in the technology integration literature in higher education. Thirteen meta-analyses, dated from 2000 to 2014 were selected to be included based on the questions asked and the presence of adequate statistical information to conduct a quantitative synthesis. The weighted…

Instrumental variable estimators hold the promise of enabling researchers to estimate the effects of educational treatments that are not (or cannot be) randomly assigned but that may be affected by randomly assigned interventions. Examples of the use of instrumental variables in such cases are increasingly common in educational and social science…

Education research rarely lends itself to large scale experimental research and true randomization, leaving the researcher to quasi-experimental designs. The problem with quasi-experimental research is that underlying factors may impact group selection and lead to potentially biased results. One way to minimize the impact of non-randomization is…

This technical paper examines the issue of attrition bias in two cohorts of the Longitudinal Surveys of Australian Youth (LSAY), based on an analysis using data from 1995 to 2002. Data up to 2002 provided eight years of information on members of the Y95 cohort and five years of information on members of the Y98 cohort. This study suggests that…

When unsubstantiated findings are used as a basis for national policy, the results can be disastrous"; the recent work by Moynihan and Mosteller, ostensibly proving" the Coleman report, actually points up its errors. (Editor/SP)

Although textbooks on educational research give only scant attention to survey research methodology its extensive use in education provides a strong rationale for improving the preparation of educational researchers in effectively applying survey methods. This study reviews methods for dealing with nonresponse bias, the primary problem presented…

A recent article by William R. Dunlap on admissions criteria is presented to illustrate applied statistical misuse. It is the basis for discussion of severe methodological problems including range truncation, dummy coding, identifying race and sex bias, measuring professional potential, and generalization of findings. The author's response is…