Background: For decades, education researchers have relied on the work of Campbell, Cook, and Shadish to help guide their thinking about valid impact estimates in the social sciences (Campbell & Stanley, 1963; Shadish et al., 2002). The foundation of this work is the "validity typology" and its associated "threats to…

In mathematics education research, proofs are often conceptualized as sequences of mathematical assertions. We argue that this ignores proofs that contain instructions to perform mathematical actions, often in the form of imperatives, which are common both in mathematical practice and in undergraduate mathematics textbooks. We consider in detail a…

Evidence-based assessment (EBA) requires that investigators employ scientific theories and research findings to guide decisions about what domains to measure, how and when to measure them, and how to make decisions and interpret results. To implement EBA, investigators need high-quality assessment tools along with evidence-based processes. We…

This methodological review highlights the trends in empirical studies where a methodological construct (i.e. verbal reports) intersects with content (i.e. literacy research). Specifically, we synthesise research on language learners' reading in which verbal reports were deployed as a methodological tool. Questioning the long-standing assumption…

American Educational Research Association (AERA) standards stipulate that researchers show evidence of the internal structure of instruments. Confirmatory factor analysis (CFA) is one structural equation modeling procedure designed to assess construct validity of assessments that has broad applicability for counselors interested in instrument…

Data analysis is a significant methodological component when conducting quantitative education studies. Guidelines for conducting data analyses in quantitative education studies are common but often underemphasize four important methodological components impacting the validity of inferences: quality of constructed measures, proper handling of…

This paper addresses the wicked problem of measuring the productivity of learning and teaching in higher education. We show how fundamental validity issues and difficulties identified in educational productivity research point to the need for a qualitatively different framework when considering the entire question. We describe the work that needs…

This article examines the diffusion and present day status of a family of unsubstantiated learning-retention myths, some of which are referred to as "the learning pyramid". We demonstrate through an extensive search in academic journals and field-specific encyclopaedias that these myths are indeed widely publicised in academia and that…

"Research Messages 2021" highlights the diverse range of research activities undertaken over the past year by the National Centre for Vocational Education Research (NCVER). In 2021, research was shaped both by the VET (vocational education and training) reform agenda, and issues emerging from the pandemic. The 2021 national VET research…

American early childhood education is in the midst of drastic change. In recent years, states have begun the process of overhauling early childhood education systems in response to federal grant competitions, bringing an increased focus on assessment and accountability for early learning programs. The assessment of young children is fraught with…

Quantitative research on sexual minority youths (SMYs) has likely contributed to misperceptions about the risk and deviance of this population. In part because it often relies on self-reported data from population-based self-administered questionnaires, this research is susceptible to misclassification bias whereby youths who are not SMYs are…

The notion of validity in the social sciences is evolving and is influenced by philosophy of science, critiques of objectivity, and epistemological debates. Methodology for validation of the knowledge claims is diverse across different philosophies of science. In other words, definition and the way to establish of validity have evolved as…

This paper addresses a relatively unexplored area in the field of learning analytics: how analytics are taken up and used as part of teaching and learning processes. Initial steps are taken towards developing design knowledge for this "middle space," with a focus on students as analytics users. First, a core set of challenges for…

With the widespread use of learning analytics (LA) tools, there is a need to explore how these technologies can be used to enhance teaching and learning. Little research has been conducted on what human processes are necessary to facilitate meaningful adoption of LA. The research problem is that there is a lack of evidence-based guidance on how…

This paper presents the results of a systematic review of agent-based modelling and simulation (ABMS) applications in the higher education (HE) domain. Agent-based modelling is a "bottom-up" modelling paradigm in which system-level behaviour (macro) is modelled through the behaviour of individual local-level agent interactions (micro).…