To support and better understand how to scale effectively, in 2020, the Millions Learning project at the Center for Universal Education (CUE) at Brookings joined the Global Partnership for Education's (GPE) Knowledge and Innovation Exchange (KIX), a joint partnership between GPE and the International Development Research Centre (IDRC), to…

Missing data are common with large scale assessments (LSAs). A typical approach to handling missing data with LSAs is the use of listwise deletion, despite decades of research showing that approach can be a suboptimal strategy resulting in biased estimates. In order to help researchers account for missing data, we provide a tutorial using R and…

Longitudinal panel studies are widely used in developmental science to address important research questions on human development across the lifespan. These studies, however, are often challenging to implement. They can be costly, time-consuming, and vulnerable to test--retest effects or high attrition over time. Planned missingness designs (PMDs),…

This paper investigates the methodological discrepancies underlying the measurement of adult learning and education (ALE) participation in the UK by focusing on four major surveys -- APiL, PIAAC, AES, and LFS. Grounded in the Total Survey Error (TSE) framework, we systematically examined the surveys' documentation and compared their definitions,…

Although there is growing social science research examining how generative AI models can be effectively and systematically applied to text-based tasks, whether and how these models can be used to analyze images remain open questions. In this article, we introduce a framework for analyzing images with generative multimodal models, which consists of…

The aim of this paper is to introduce a software that is appropriate for the generalizability theory for not only balanced but also unbalanced data sets. Because it is possible to have unbalanced data sets while conducting a study, the researchers have devised an easy solution, other than deleting data, to balance the design to cope with this…

States' commitments to tackling long standing inequities have been stifled by missing data, long delays, insufficient data-analysis tools, and the excessive reporting burden placed on states and institutions. If states hope to achieve their completion and equity goals, they need access to data that does not leave them guessing--so they can…

Meta-analysis is a widely used methodology to combine evidence from different sources examining a common research phenomenon, to obtain a quantitative summary of the studied phenomenon. In the medical field, multiple studies investigate the effectiveness of new treatments and meta-analysis is largely performed to generate the summary (average)…

While conflict event data sets are increasingly used in contemporary conflict research, important concerns persist regarding the quality of the collected data. Such concerns are not necessarily new. Yet, because the methodological debate and evidence on potential errors remains scattered across different subdisciplines of social sciences, there is…

Large-scale analyses are a powerful and increasingly common tool for investigating a range of public health and social concerns (Pienta, O'Rourke, & Franks, 2011). This series will provide a primer on large-scale secondary analysis in school psychology, with this article focusing on considerations for researchers interested in applying and…

Missing data result in less precise and possibly biased effect estimates in single studies. Bias arising from studies with incomplete outcome data is naturally propagated in a meta-analysis. Conventional analysis using only individuals with available data is adequate when the meta-analyst can be confident that the data are missing at random (MAR)…

Qualitative research has been strongly affected by the COVID-19 pandemic, highlighting the possibilities that Voice over Internet Protocol (VoIP) technologies such as Skype, WhatsApp, and Zoom offer to qualitative scholars. Based on the experience of using such technologies to collect qualitative data for our PhD studies, we present how we dealt…

Social science datasets usually have missing cases, and missing values. All such missing data has the potential to bias future research findings. However, many research reports ignore the issue of missing data, only consider some aspects of it, or do not report how it is handled. This paper rehearses the damage caused by missing data. The paper…

It is widely believed that regression models for binary responses are problematic if we want to compare estimated coefficients from models for different groups or with different explanatory variables. This concern has two forms. The first arises if the binary model is treated as an estimate of a model for an unobserved continuous response and the…

Realist evaluation (RE) is a research design increasingly used in program evaluation, that aims to explore and understand the influence of context and underlying mechanisms on intervention or program outcomes. Several methodological challenges, however, are associated with this approach. This article summarizes RE key principles and examines some…