Concerns about the conduct of research are pervasive in many fields, including education. In this preregistered study, we replicated and extended previous studies from other fields by asking education researchers about 10 questionable research practices and five open research practices. We asked them to estimate the prevalence of the practices in…

A substantial number of randomized trials of educational interventions that have been conducted over the past two decades have produced null results, with either no impact or an unreliable estimate of impact on student achievement or other outcomes of interest. The investment of time and money spent implementing such trials warrants more useful…

As a result of the evidence-based decision-making movement, the number of randomized trials evaluating educational programs and curricula has increased dramatically over the past 20 years. Policy makers and practitioners are encouraged to use the results of these trials to inform their decision making in schools and school districts. At the same…

In this response, we first show that Simpson's proposed analysis answers a different and less interesting question than ours. We then justify the choice of prior for our Bayes factors calculations, but we also demonstrate that the substantive conclusions of our article are not substantially affected by varying this choice.

The advent of online platforms such as Amazon's Mechanical Turk (MTurk) has expanded considerably researchers' options for collecting research data. Many researchers, however, express understandable skepticism of the viability of using platforms such as MTurk. In this article, we provide a background on the use of MTurk as a mechanism for…

A recent paper uses Bayes factors to argue a large minority of rigorous, large-scale education RCTs are "uninformative." The definition of "uninformative" depends on the authors' hypothesis choices for calculating Bayes factors. These arguably overadjust for effect size inflation and involve a fixed prior distribution,…

In the current article, we consider the influential position of the Programme for International Student Assessment (PISA) and discuss several methodological areas that demonstrate the need for caution when using and interpreting PISA results. We motivate our argument by briefly describing the program's increased influence in educational policy…

Design-based research (DBR) evolved near the beginning of the 21st century and was heralded as a practical research methodology that could effectively bridge the chasm between research and practice in formal education. In this article, the authors review the characteristics of DBR and analyze the five most cited DBR articles from each year of this…

The solution to the lack of productivity of educational research is the recognition that applied research can be scientific and the reexamination of the implications of the scientific orientation for research strategies. (Author/EB)

Discusses the role of research methodology as disciplined inquiry in educational research. Uses examples to address issues of generalizability of results, precision, and choice of research methods. (MK)

A major source of confusion in educational research comes when qualitative research is regarded as if it were one approach. Qualitative research has many varieties that can be identified and understood by using the notion of research traditions. Six approaches from the social and behavioral sciences are compared. (VM)

Outlines the recent transition, within educational research, from conflict to cooperation between quantitative and qualitative approaches. Asserts that compatibility is based on a confusion between method as technique and method as logic of justification. Argues that the claim of compatibility cannot be sustained and blocks an interesting and…

Argues that defining "progress" is fundamental to understanding how effective educational research is in improving practice. Discusses the problem of partial specification and its consequences for defining the research-practice relationship. Presents three different, but complementary, approaches to defining "progress":…

Examines data analysis at three levels: analysis of data; secondary analysis is the re-analysis of data for the purpose of answering the original research question with better statistical techniques, or answering new questions with old data; and, meta-analysis refers to the statistical analysis of many analysis results from individual studies for…

Relativism is an inevitable consequence of our interpretive mode of being in the world. This is so for both our daily lives and our professional lives. What does not overly concern us at the former level should likewise not concern us at the latter level. (Author/BJV)