This study evaluated differential item functioning (DIF) in achievement motivation items before and after using anchoring vignettes as a statistical tool to account for group differences in response styles across gender and ethnicity. We applied the nonparametric scoring of the vignettes to motivation items from the 2015 Programme for…

The aim of the current research was to provide recommendations to facilitate the development and use of anchoring vignettes (AVs) for cross-cultural comparisons in education. Study 1 identified six factors leading to order violations and ties in AV responses based on cognitive interviews with 15-year-old students. The factors were categorized into…

In the context of Programme for International Student Assessment (PISA), paradoxical findings concerning the relationship between questionnaire scales and student achievement are often documented. These questionnaire scales are found to correlate positively with student achievement within countries at the individual level, but negatively at the…

Empirical evidence suggests children's socio-emotional skills--an important determinant of school achievement--vary according to socioeconomic family background. This study assesses the degree to which differences in socio-emotional skills contribute to the achievement gap between socioeconomically advantaged and disadvantaged children. We used…

We investigate whether Anchoring Vignettes (AV) improve intercultural comparability of non-cognitive student-directed factors (e.g., procrastination). So far, correlation analyses for anchored and non-anchored scores with a criterion have been used to demonstrate the effectiveness of AV in improving data quality. However, correlation analyses are…

International large-scale assessments such as the Programme for International Student Assessment (PISA) yield comparative indicators of student achievement in various competence domains. This article focuses on global competence as a suggested cross-curricular domain for the PISA 2018 study. The measurement of global competence is related to a…

We examine construct validity of the anchoring method used with 12 noncognitive scales from the Programme for International Student Assessment (PISA) 2012 project. This method combines individuals' responses to vignettes and self-rated scores based on Likert-type items. It has been reported that the use of anchoring vignettes can reverse…

Two different tools for assessing pedagogical content knowledge (PCK) of mathematics teachers used in the framework of the COACTIV study are systematically compared in this paper, namely the paper-and-pencil test consisting of items on the three facets "knowledge of explaining and representation," "knowledge of student thinking and…

Secondary data analyses of large-scale international student assessment at the country level often reveal that educational systems which are high in academic achievement are found low in attitudinal outcomes of schooling, and vice versa. This is contrary to the non-negative attitude-achievement relationship frequently found at the student…

Multiple-source inquiry that involves collaboration and deliberation is a complex construct; accordingly, valid measurement of these related competencies should require students to demonstrate their collaborative inquiry and social deliberation skills as part of an integrated performance. In this article, we report findings from the first phase of…

Self-reports are an indispensable source of information in education research but they are often affected by heterogeneity in reporting behavior. Failing to correct for this heterogeneity can lead to invalid comparisons across groups. The researchers use the parametric anchoring vignette method to correct for cross-country incomparability of…

Large-scale assessment, such as the Programme for International Assessment (PISA), plays an increasingly important role in current educational practice and politics. However, many scholars have questioned the validity and reliability of the tests and the extent to which they actually constitute trustworthy representations of students' knowledge.…

"Deep reading comprehension" refers to the process required to succeed at tasks defined by the Common Core State Literacy Standards, as well as to achieve proficiency on the more challenging reading tasks in the Program for International Student Assessment (PISA) framework. The purpose of this study was to test the hypothesis that three…

Since 2003, international project PISA evaluates 15-year old students in solving problems that include "decision taking", "analysis and design of systems" and "trouble-shooting". This article presents the results of a pilot research conducted with 215 students from first to fourth grade of a high school in Sarajevo…