Researchers increasingly have access to two types of data: (i) large observational datasets where treatment (e.g., class size) is not randomized but several primary outcomes (e.g., graduation rates) and secondary outcomes (e.g., test scores) are observed and (ii) experimental data in which treatment is randomized but only secondary outcomes are…

In this article, the authors address some of the scientific challenges associated with using observation as a research method. The authors ask how researchers contextualise and understand observation in terms of its theoretical underpinnings and how it is conducted. Using a vignette in the kindergarten context, the authors explore how observation…

In the early months of the COVID-19 pandemic, many research funding organizations were faced with the choice of suspending their peer review panels, or else continuing their decision-making processes virtually. Although seen part of a longer drive to make peer review more cost and time efficient as well as to combat climate and sustainability…

In this article, we reflect on our experiences with teaching ethnographic skills and sensibilities to MSc students at a Dutch university. Using methods such as (self)observation, journaling and reflection, we highlight dilemmas (conceptual, practical, ethical) faced by students when doing ethnographic fieldwork and dilemmas that emerged in…

This paper explores the challenges of filming as a qualitative research method in practical school subjects. We exemplify with an observation study using video data collected from lessons in Home Economics, but the issues we discuss are also relevant for other subjects with practical elements, such as Arts, Crafts, Music, Physical Education, and…

The growing interest in video research and new technologies for recording human interaction has stirred debates about intrusiveness and 'reactivity' understood as researcher-derived changes in subjects. In addition to a plethora of concepts referring to such effects in extant literature, different ontological and epistemological positions provide…

Context: A perennial problem in applied statistics is the inability to justify strong claims about cause-and-effect relationships without full knowledge of the mechanism determining selection into treatment. Few research designs other than the well-implemented random assignment study meet this requirement. Researchers have proposed partial…

The National Institute of Mental Health created the National Database for Autism Research (NDAR) to accelerate autism knowledge through data sharing and collaboration. However, our experience using NDAR reveals systematic challenges across several aspects of data submission, selection, management, and analysis that limit utility of this resource.…

In the complex world of educational policy, causal inferences will be debated. As we review non-experimental designs in educational policy, we focus on how to clarify and focus the terms of debate. We begin by presenting the potential outcomes/counterfactual framework and then describe approximations to the counterfactual generated from the…