This paper argues for three aspects of tolerance with respect to QCA research: tolerance with respect to different approaches to QCA; producing QCA research with tolerance (work that is resistant to criticism); and for QCA researchers to be clear about the tolerance of the solutions they present -- especially in terms of calibration and truth…

The improvement of existing science education interventions and the development of new ones requires cohesive, large-scale evaluation and testing that inform what works, for whom, and in what context. Systems approaches to design, implement, and evaluate interventions require knowledge, skills, and expertise in studying humans alongside the lived…

This reflective piece tells the story of how I started out doing Conversation Analysis (CA) and have been transitioning into doing mixed methods for some years now. My basic argument is that language learning talk is too complex a phenomenon to analyse using a single methodology. Specifically, it is extremely difficult to isolate from the…

Assessment and institutional research offices have too much data and too little time. Standard reporting often crowds out opportunities for innovative research. Fortunately, advancements in data science now offer a clear solution. It is equal parts technique and philosophy. The first and easiest step is to modernize data work. This column…

The role of knowledge and reading comprehension has seen a recent explosion of attention from researchers, journalists, and policy advocates. Much of this discourse describes knowledge in neutral terms such as knowledge of "the world". That knowledge of the world, however, is woven into the fabric of the gendered world we live in and its…

This note is concerned with evaluation of location parameters for polytomous items in multiple-component measuring instruments. A point and interval estimation procedure for these parameters is outlined that is developed within the framework of latent variable modeling. The method permits educational, behavioral, biomedical, and marketing…

Over the past two decades, several methods have been developed to evaluate the societal impact of research. Compared to the practical development of the field, the conceptual development is relatively weak. This review article contributes to the latter by elucidating the theoretical aspects of the dominant methods for evaluating societal impact of…

Conventional advice discourages controlling for postoutcome variables in regression analysis. By contrast, we show that controlling for commonly available postoutcome (i.e., future) values of the treatment variable can help detect, reduce, and even remove omitted variable bias (unobserved confounding). The premise is that the same unobserved…

This article argues that Agent-Based Modelling, owing to its capabilities and methodology, has a distinctive contribution to make to delivering coherent social science prediction. The argument has four parts. The first identifies key elements of social science prediction induced from real research across disciplines, thus avoiding a straw person…

Increased charter school demand creates a critical need for reliable information on outcomes. Stanford's Center for Research on Education Outcomes (CREDO) has a series of influential reports on charter schools using the virtual control record (VCR) method. However, the VCR method has been criticized and the validity of CREDO's findings challenged,…

A reproducible analysis is one in which an independent entity, using the same data and the same statistical code, would obtain the exact same result as the previous analyst. Reproducible analyses utilize script-based analyses and open data to aid in the reproduction of the analysis. A reproducible analysis does not ensure the same results are…

Learning progressions (LPs) are cognitive models that describe the development of scientific knowledge and practices in students. They are constructed based on learning theories and student data. Scholars have advocated for using LPs to align curriculum, instruction, and assessment into a coherent system, and by doing so, promote productive…

This paper looks at the tension between the desire to claim predictive ability for Agent-Based Models (ABMs) and its extreme difficulty for social and ecological systems, suggesting that this is the main cause for the continuance of a rhetoric of prediction that is at odds with what is achievable. Following others, it recommends that it is better…

Meta-analyses of educational research findings frequently involve statistically dependent effect size estimates. Meta-analysts have often addressed dependence issues using ad hoc approaches that involve modifying the data to conform to the assumptions of models for independent effect size estimates, such as by aggregating estimates to obtain one…

Debates surrounding the use of data science in educational AI are frequently rather entrenched, revolving around commercial models and talk of teacher replacement. This article explores the potential for digital textual analysis within humanities and social science education, advocating for a sociologically-driven approach that complements, rather…