This study examines potential assessment bias based on students' primary language status in PISA 2018. Specifically, multilingual (MLs) and nonmultilingual (non-MLs) students in the United States are compared with regard to their response time as well as scored responses across three cognitive domains (reading, mathematics, and science).…

Cross-national comparisons of educational achievement rely upon each participating country collecting nationally representative data. While obtaining high response rates is a key part of reaching this goal, other potentially important factors may also be at play. This paper focuses on one such issue--exclusion rates--which has received relatively…

This colloquium brings information about a second cycle of OECD's International Early Learning and Well-being Study (IELS) to the early childhood community, and offers a further critique of the approach to comparative research that the IELS embodies.

Well-being 2030 has become the latest rationale for the OECD's education work. This vision has given rise to new assessments of student well-being beginning with PISA 2015. The OECD, recognising the problems of PISA 2015, conceptualised a wider student well-being construct in PISA 2018, and attempted to measure 'students' feelings'. However,…

This article addresses conceptual and methodological shortcomings regarding conducting and interpreting intelligence test factor analytic research that appeared in the Decker, S. L., Bridges, R. M., Luedke, J. C., & Eason, M. J. (2020). Dimensional evaluation of cognitive measures: Methodological confounds and theoretical concerns.…

Following the onset of the novel coronavirus disease 2019 (COVID-19) pandemic, daily life significantly changed for the population. Accordingly, researchers interested in examining patterns of change over time may now face discontinuities around the pandemic. Researchers collecting in-person longitudinal data also had to cancel or delay data…

This article examines whether the way that PISA models item outcomes in mathematics affects the validity of its country rankings. As an alternative to PISA methodology, a two-parameter logistic model is applied to PISA mathematics item data from Italy and Spain for the year 2009. In the estimation procedure, item difficulty and dispersion…

Data analysis is a significant methodological component when conducting quantitative education studies. Guidelines for conducting data analyses in quantitative education studies are common but often underemphasize four important methodological components impacting the validity of inferences: quality of constructed measures, proper handling of…

In studies of educational achievement, students' self-reported number of books in the family home is a frequently used proxy for social, cultural, and economic background. Absent hard evidence about what this variable captures or how well, its use has been motivated by strong associations with student outcomes. I show that these associations rest…

Large-scale assessments (LSAs) use Mislevy's "plausible value" (PV) approach to relate student proficiency to noncognitive variables administered in a background questionnaire. This method requires background variables to be completely observed, a requirement that is seldom fulfilled. In this article, we evaluate and compare the…

In the current article, we consider the influential position of the Programme for International Student Assessment (PISA) and discuss several methodological areas that demonstrate the need for caution when using and interpreting PISA results. We motivate our argument by briefly describing the program's increased influence in educational policy…

Purpose: The purpose of this paper is to develop a framework for reconceptualising research on school autonomy to redress the limitations of traditional research, strengthen the conceptual links between school autonomy and learning outcomes and offer a range of new strategies for studying the interplay of school autonomy, leadership and learning.…

Bui, Craig, and Imberman assessed the impact of gifted and talented programs on student achievement using regression discontinuity and random assignment to gifted magnet schools. In both analyses, they found minimal impact of the gifted and talented programs on student achievement. This commentary addresses two concerns associated with the study.…

Survey research in education is a popular quantitative research design and is used to determine opinions, attitudes, behavior, habits, and perceptions on a wide variety of topics. While many people are familiar with survey methods, it is often used in a 'quick and dirty' way to get information. This case aims to describe some of the challenges of…

Purpose: This policy brief, the third in the AERI-NEPC eBrief series "Understanding validity issues around the world", discusses validity issues surrounding International Large Scale Assessment (ILSA) programs. ILSA programs, such as the well-known Programme of International Student Assessment (PISA) and the Trends in International…