Single-case experimental designs (SCEDs) have a long history in clinical and educational disciplines. One underdeveloped area in advancing SCED design and analysis is understanding the process of how internal validity threats and operational concerns are avoided or mitigated. Two strategies to ameliorate such issues in SCED involve replication and…

Randomized response techniques (RRTs) are useful survey tools for estimating the prevalence of sensitive issues, such as the prevalence of doping in elite sports. One type of RRT, the unrelated question model (UQM), has become widely used because of its psychological acceptability for study participants and its favorable statistical properties.…

In this essay, we focus on methodological issues that reviewers and editors commonly encounter when evaluating empirical articles in scholarship of teaching and learning in management education. We organize our discussion around three stages--design, analysis, and reporting. The essay identifies which types of issues are likely to receive…

Longitudinal panel studies are widely used in developmental science to address important research questions on human development across the lifespan. These studies, however, are often challenging to implement. They can be costly, time-consuming, and vulnerable to test--retest effects or high attrition over time. Planned missingness designs (PMDs),…

In the field of educational technology and instructional design, research methods are emerging that aim to curate different forms of knowledge and insights beyond traditional research studies, or what Reigeluth and An (in Reigeluth and Carr-Chellman (eds) Instructional-design theories and models: Building a common knowledge base, Lawrence Erlbaum…

Realist evaluation (RE) is a research design increasingly used in program evaluation, that aims to explore and understand the influence of context and underlying mechanisms on intervention or program outcomes. Several methodological challenges, however, are associated with this approach. This article summarizes RE key principles and examines some…

As a research field geared toward understanding and improving learning, Learning Analytics (LA) must be able to provide empirical support for causal claims. However, as a highly applied field, tightly controlled randomized experiments are not always feasible nor desirable. Instead, researchers often rely on observational data, based on which they…

The aim of this paper is to introduce a software that is appropriate for the generalizability theory for not only balanced but also unbalanced data sets. Because it is possible to have unbalanced data sets while conducting a study, the researchers have devised an easy solution, other than deleting data, to balance the design to cope with this…

Particular ethical tensions and dilemmas emerge when conducting qualitative educational research. This is further compounded where the predominant approach to research ethics is underscored by a dominant principalism which construes ethical rules as both universal and absolute. This article focuses on the 'perfect ethical storm' which is arguably…

Experimental studies are often employed to test the effectiveness of teaching innovations such as new pedagogy, curriculum, or learning resources. This article offers guidance on good practice in developing research designs, and in drawing conclusions from published reports. Random control trials potentially support the use of statistical…

Single-case researchers frequently adopt a form of response-guided experimentation where decisions about the design of the study are made based on an ongoing visual analysis. For example, multiple-baseline researchers may delay the start of intervention until data document a stable baseline pattern so that baseline trends can be reliably extended,…

Researchers sometimes conduct a study and find that the predicted relation between variables did not exist or that the intervention did not have a positive impact on student outcomes; these are referred to as null findings because they fail to disconfirm the null hypothesis. Rather than consider such studies as failures and disregard the null…

Treatment effect estimates from a "regression discontinuity design" (RDD) have high internal validity. However, the arguments that support the design apply to a subpopulation that is narrower and usually different from the population of substantive interest in evaluation research. The disconnect between RDD population and the…

This paper presents best practices for conducting survey research using Amazon Mechanical Turk (MTurk). Readers will learn the benefits, limitations, and trade-offs of using MTurk as compared to other recruitment services, including SurveyMonkey and Qualtrics. A synthesis of survey design guidelines along with a sample survey are presented to help…

This article presents an overview of internet-based research, highlighting the absence of a standard terminology to define and classify such research. The label internet-based research or online research can cover a diverse range of research designs and methods, involving different degrees of ethical concern regarding privacy, transparency,…