The process of quantification is a powerful development shaping many domains of life today. In the area of education, for example, performance measurement, testing and ranking have become common tools of governance. Quantification is not a neutral way of describing society, but a process of valorisation. It has three sociologically relevant…

Data analysis is a significant methodological component when conducting quantitative education studies. Guidelines for conducting data analyses in quantitative education studies are common but often underemphasize four important methodological components impacting the validity of inferences: quality of constructed measures, proper handling of…

Purpose: Fabricated data jeopardize the reliability of large-scale population surveys and reduce the comparability of such efforts by destroying the linkage between data and measurement constructs. Such data result in the loss of comparability across participating countries and, in the case of cyclical surveys, between past and present surveys.…

Purpose: This policy brief, the third in the AERI-NEPC eBrief series "Understanding validity issues around the world", discusses validity issues surrounding International Large Scale Assessment (ILSA) programs. ILSA programs, such as the well-known Programme of International Student Assessment (PISA) and the Trends in International…

This paper examines how international, large-scale skills assessments (ILSAs) engage with the broader societies they seek to serve and improve. It looks particularly at the discursive work that is done by different interest groups and the media through which the findings become part of public conversations and are translated into usable form in…

This article presents findings on the consequences of matrix sampling of context questionnaires for the generation of plausible values in large-scale assessments. Three studies are conducted. Study 1 uses data from PISA 2012 to examine several different forms of missing data imputation within the chained equations framework: predictive mean…

Purpose: This paper presents a moderated discussion on popular misconceptions, benefits and limitations of International Large-Scale Assessment (ILSA) programs, clarifying how ILSA results could be more appropriately interpreted and used in public policy contexts in the USA and elsewhere in the world. Design/methodology/approach: To bring key…

Background: In the context of international large scale assessments, it is often not feasible to implement a complete survey of all relevant populations. For example, the OECD Program for International Student Assessment surveys both students and schools, but does not obtain information from teachers. In contrast the OECD Teaching and Learning…

This paper briefly discusses four measurement challenges of data science or "big data" in educational assessments that are enabled by technology: 1. Dealing with change over time via time-based data. 2. How a digital performance space's relationships interact with learner actions, communications and products. 3. How layers of…

Results from international large-scale assessments (ILSAs) garner considerable attention in the media, academia, and among policy makers. Although there is widespread recognition that ILSAs can provide useful information, there is debate about what types of comparisons are the most meaningful and what could be done to assure more sound…

This article extends a methodological approach considered by Bolt and Johnson for the measurement and control of extreme response style (ERS) to the analysis of rating data from multiple scales. Specifically, it is shown how the simultaneous analysis of item responses across scales allows for more accurate identification of ERS, and more effective…

Not much is known about the ways in which assessment data have actually been used in education policy to date. Understanding the role of assessments in informing system-level decision-making is a first step towards helping stakeholders improve the design and usefulness of assessments. Moreover, this understanding can help to further discussions…

This Report Card presents an overview of inequalities in child well-being in 41 countries of the European Union (EU) and the Organisation for Economic Co-operation and Development (OECD). It focuses on 'bottom-end inequality' -- the gap between children at the bottom and those in the middle -- and addresses the question 'how far behind are…

This report provides a systematic review and empirical evidence related to the experiences of middle-income countries and economies participating in the Programme for International Student Assessment (PISA), 2000 to 2015. PISA is a triennial survey that aims to evaluate education systems worldwide by testing the skills and knowledge of 15-year-old…

A general diagnostic model was used to specify and compare two multidimensional item-response-theory (MIRT) models for longitudinal data: (a) a model that handles repeated measurements as multiple, correlated variables over time (Andersen, 1985) and (b) a model that assumes one common variable over time and additional orthogonal variables that…