Missing data are common with large scale assessments (LSAs). A typical approach to handling missing data with LSAs is the use of listwise deletion, despite decades of research showing that approach can be a suboptimal strategy resulting in biased estimates. In order to help researchers account for missing data, we provide a tutorial using R and…

This study examines potential assessment bias based on students' primary language status in PISA 2018. Specifically, multilingual (MLs) and nonmultilingual (non-MLs) students in the United States are compared with regard to their response time as well as scored responses across three cognitive domains (reading, mathematics, and science).…

Cross-national comparisons of educational achievement rely upon each participating country collecting nationally representative data. While obtaining high response rates is a key part of reaching this goal, other potentially important factors may also be at play. This paper focuses on one such issue--exclusion rates--which has received relatively…

While methodologies are often presented as standardised procedures which, under specified conditions, should lead to the same conclusions, this case study presents the complex and deeply personal process of research (Addey and Piattoeva 2022). We analyse our application of network ethnography -- an approach presented by Ball (2016) -- to the study…

This colloquium brings information about a second cycle of OECD's International Early Learning and Well-being Study (IELS) to the early childhood community, and offers a further critique of the approach to comparative research that the IELS embodies.

Well-being 2030 has become the latest rationale for the OECD's education work. This vision has given rise to new assessments of student well-being beginning with PISA 2015. The OECD, recognising the problems of PISA 2015, conceptualised a wider student well-being construct in PISA 2018, and attempted to measure 'students' feelings'. However,…

As the international large-scale assessments (ILSAs) become more popular, policy makers and education practitioners are interested in collecting as much student background information as possible to better understand the learning context of their students. To collect such abundant information, administrators need to develop a lot of questions.…

Computer-based administration of large-scale assessments makes it possible to collect a rich set of information on test takers, through analysis of the log files recording interactions between the computer interface and the server. This report examines timing and engagement indicators from the Survey of Adult Skills, a product of the Programme for…

This article examines whether the way that PISA models item outcomes in mathematics affects the validity of its country rankings. As an alternative to PISA methodology, a two-parameter logistic model is applied to PISA mathematics item data from Italy and Spain for the year 2009. In the estimation procedure, item difficulty and dispersion…

Data analysis is a significant methodological component when conducting quantitative education studies. Guidelines for conducting data analyses in quantitative education studies are common but often underemphasize four important methodological components impacting the validity of inferences: quality of constructed measures, proper handling of…

In studies of educational achievement, students' self-reported number of books in the family home is a frequently used proxy for social, cultural, and economic background. Absent hard evidence about what this variable captures or how well, its use has been motivated by strong associations with student outcomes. I show that these associations rest…

Large-scale assessments (LSAs) use Mislevy's "plausible value" (PV) approach to relate student proficiency to noncognitive variables administered in a background questionnaire. This method requires background variables to be completely observed, a requirement that is seldom fulfilled. In this article, we evaluate and compare the…

In the current article, we consider the influential position of the Programme for International Student Assessment (PISA) and discuss several methodological areas that demonstrate the need for caution when using and interpreting PISA results. We motivate our argument by briefly describing the program's increased influence in educational policy…

Purpose: The purpose of this paper is to develop a framework for reconceptualising research on school autonomy to redress the limitations of traditional research, strengthen the conceptual links between school autonomy and learning outcomes and offer a range of new strategies for studying the interplay of school autonomy, leadership and learning.…

Survey research in education is a popular quantitative research design and is used to determine opinions, attitudes, behavior, habits, and perceptions on a wide variety of topics. While many people are familiar with survey methods, it is often used in a 'quick and dirty' way to get information. This case aims to describe some of the challenges of…