Although self-report surveys are widely used for data collection, data quality can vary across populations because certain groups are more likely to engage in insufficient effort responding (IER). Our study examined how different levels of the educational system--student groups, schools, and cultural contexts--affect data quality due to IER, using…

The present paper questions the content validity of the eight creativity-related self-report scales available in PISA 2022's context questionnaire and provides a set of considerations for researchers interested in using these indexes. Specifically, we point out some threats to the content validity of these scales (e.g., "creative thinking…

Creative activities are typically thought to be positively associated with creative ability, whether because more creative individuals select into creative activities, or because those activities support the development of creativity, or both. However, the PISA 2022 creative thinking report revealed an unexpected finding: Creative ability was…

Questionnaires are by far the most common tool for measuring noncognitive constructs in psychology and educational sciences. Response bias may pose an additional source of variation between respondents that threatens validity of conclusions drawn from questionnaire data. We present a mixture modeling approach that leverages response time data from…

This study aims to assess student motivation to provide valid responses to the PISA student questionnaire. This was done by modeling response times using a three-component finite mixture model, comprising two satisficing response styles (rapid and idle) and one optimizing response style. Each participant's motivation was operationalized as their…

The Skills Imperative 2035 is a five-year strategic research programme which is investigating future skills needs, skills supply and skill development, with a particular focus on the 'Essential Employment Skills' (EES) that are projected to be most vital across the labour market in 2035. The focus in this stage of The Skills Imperative 2035 is on…

Purpose: Evidence from past surveys suggests that some interviewees simplify their responses even in very well-organized and highly respected surveys. This paper aims to demonstrate that some interviewers, too, simplify their task by at least partly fabricating their data, and that, in some survey research institutes, employees simplify their task…

The Skills Imperative 2035 is a five-year strategic research programme which is investigating future skills needs, skills supply and skill development, with a particular focus on the 'Essential Employment Skills' (EES) that are projected to be most vital across the labour market in 2035. This paper investigates childhood skill development and…

Large-scale assessments (LSAs) use Mislevy's "plausible value" (PV) approach to relate student proficiency to noncognitive variables administered in a background questionnaire. This method requires background variables to be completely observed, a requirement that is seldom fulfilled. In this article, we evaluate and compare the…

The Programme for International Student Assessment (PISA) assesses the knowledge and skills in mathematics, reading and science of 15-year-old pupils in countries around the world. PISA is run by the Organisation for Economic Co-operation and Development (OECD), and assessment is typically undertaken every 3 years, allowing us to chart how…

School climate is one of the significant factors determining educational achievement. However, the lack of instruments to measure it has complicated the study of this concept in Russia. We review the history of the study of the concept of "school climate," and we discuss approaches to how it can be defined. We describe the most widely…

Background: This paper extends a recent study by Kaplan and Su ("J Educ Behav Stat" 41: 51-80, 2016) examining the problem of matrix sampling of context questionnaire scales with respect to the generation of plausible values of cognitive outcomes in large-scale assessments. Methods: Following Weirich et al. ("Nested multiple…

Cognitive interviewing (CI) provides a method of systematically collecting validity evidence of response processes for questionnaire items. CI involves a range of techniques for prompting individuals to verbalise their responses to items. One such technique is concurrent verbalisation, as developed in Think Aloud Protocol (TAP). This article…

International large-scale assessments such as the Programme for International Student Assessment (PISA) yield comparative indicators of student achievement in various competence domains. This article focuses on global competence as a suggested cross-curricular domain for the PISA 2018 study. The measurement of global competence is related to a…

Science excellence is associated not only with a student's inherent aptitude but also a range of contextual factors. The objective of this paper was to identify the most important contextual characteristics of top performers in scientific literacy, by simultaneously considering factors at the PISA questionnaire-based student, family, and school…