Background/Context: Single-case experiment designs (SCEDs) are experimental designs in which a small number of cases are repeatedly measured over time, with manipulation of baseline and intervention phases. Because SCEDs often rely on direct behavioral observations, count data are common. To account for both the clustering and the non-normal…

Large-scale survey assessments (LSAs) such as NAEP, TIMSS, PIRLS, IELS, and NAPLAN produce plausible values of student proficiency for estimating population statistics. Plausible values are imputed values for latent proficiency variables. While prominently used for LSAs, they are applicable to a wide range of latent variable modelling contexts…

Background: Double/debiased machine learning (DML) methods have been proposed to overcome the regularization bias from the naive approach of ML methods (Chernozhukov et al., 2018). DML methods use a partialling-out approach which removes the effect of confounders from both the treatment and outcome and then regresses the residualized outcome on…

Large-scale assessments are rich sources of data that can inform a diverse range of research questions related to educational policy and practice. For this reason, datasets from large-scale assessments are available to enable secondary analysts to replicate and extend published reports of assessment results. These datasets include multiple imputed…

Causal mediation analysis has gained increasing attention in recent years. This article guides empirical researchers through the concepts and challenges of causal mediation analysis. I first clarify the difference between traditional and causal mediation analysis and highlight the importance of adjusting for the treatment-by-mediator interaction…

School start regulations allocate children born immediately either side of a given date to different life paths: those slightly older starting school a full year earlier. School effectiveness literature exploits this to estimate causal effects described as 'the absolute effect of schooling' or 'the effect of an additional year's schooling', using…

We consider the cascading effects of researcher decisions throughout the process of quantifying racially polarized voting (RPV). We contrast three methods of estimating precinct racial composition, Bayesian Improved Surname Geocoding (BISG), fully Bayesian BISG, and Citizen Voting Age Population (CVAP), and two algorithms for performing ecological…

Background and Context: Data science and statistics are used across a broad spectrum of professions, experience levels and programming languages. The popular scientific computing languages, such as Matlab, Python and R, were organized without using empirical methods to show evidence for or against their design choices, resulting in them feeling…

Background: Multisite designs, also known as blocked designs, are experimental designs in which the random assignment of treatment and control conditions is within each site (or block) after the random selection of sites (or blocks). Multisite designs exhibit remarkable adaptability and, statistically, it can maintain a rigorous basis for…

Background: One of the most vexing threats to education field trials is attrition--when some subjects drop out before it is complete. Since outcomes are not available for subjects who attrit, they are typically dropped from any analysis estimating average effects on the outcome. However, since attrition may itself have been affected by treatment…

Visual analysis is the primary methodology used to determine treatment effects from graphed single-case design data. Previous studies have demonstrated mixed findings related to interrater agreement between both expert and novice visual analysts, which represents a critical limitation of visual analysis and supports calls for also presenting…

This article presents methods for using extant data to improve the properties of estimators of the standardized mean difference (SMD) effect size. Because samples recruited into education research studies are often more homogeneous than the populations of policy interest, the variation in educational outcomes can be smaller in these samples than…

Most methods for structural equation modeling (SEM) focused on the analysis of covariance matrices. However, "Historically, interesting psychological theories have been phrased in terms of correlation coefficients." This might be because data in social and behavioral sciences typically do not have predefined metrics. While proper methods…

Research in education and behavioral sciences often involves the use of latent variable models that are related to indicators, as well as related to covariates or outcomes. Such models are subject to interpretational confounding, which occurs when fitting the model with covariates or outcomes alters the results for the measurement model. This has…

To design effective instruction, educators need to know "what" design strategies are generally effective and why these strategies work, based on the mechanisms through which they operate. Experimental comparison studies, which compare one instructional design against another, can generate much needed evidence in support of effective…