International assessments inform education policy debates, yet little is known about their floor effects: To what extent do they fail to differentiate between the lowest performers, and what are the implications of this? TIMSS, SACMEQ, and LLECE data are analyzed to answer this question. In TIMSS, floor effects have been reduced through the…

A psychological framework for different types of items commonly used with mixed-format exams is proposed. A choice model based on signal detection theory (SDT) is used for multiple-choice (MC) items, whereas an item response theory (IRT) model is used for open-ended (OE) items. The SDT and IRT models are shown to share a common conceptualization…

In this research, the effect of an item's type and cognitive level on its difficulty index was investigated. The data source of the study consisted of the responses of the 12535 students in the Turkey sample (6079 and 6456 students from eighth and fourth grade respectively) of TIMSS 2015. The responses were a total of 215 items at the eighth-grade…

The purpose of this study was to examine eighth-grade students' science performance in terms of two test design components, item format, and cognitive domain. The portion of Taiwanese data came from the 2011 administration of the Trends in International Mathematics and Science Study (TIMSS), one of the major international large-scale assessments…

Wording effects associated with positively and negatively worded items have been found in many scales. Such effects may threaten construct validity and introduce systematic bias in the interpretation of results. A variety of models have been applied to address wording effects, such as the correlated uniqueness model and the correlated traits and…

TIMSS (Trends in International Mathematics and Science Study) is a long-standing international assessment of mathematics and science at the fourth and eighth grades that has been collecting trend data every four years since 1995. About 70 countries use TIMSS trend data for monitoring the effectiveness of their education systems in a global…

The chapters in this online volume comprise the TIMSS & PIRLS International Study Center's technical report of the methods and procedures used to develop, implement, and report the results of TIMSS 2019. There were various technical challenges because TIMSS 2019 was the initial phase of the transition to eTIMSS, with approximately half the…

Many scales contain both positively and negatively worded items. Reverse recoding of negatively worded items might not be enough for them to function as positively worded items do. In this study, we commented on the drawbacks of existing approaches to wording effect in mixed-format scales and used bi-factor item response theory (IRT) models to…

Traditionally, women and minorities have not been fully represented in science and engineering. Numerous studies have attributed these differences to gaps in science achievement as measured by various standardized tests. Rather than describe mean group differences in science achievement across multiple cultures, this study focused on an in-depth…

Disentangled sources of differential item functioning (DIF) in a multilanguage assessment for which multiple factors were expected to be causing DIF. Data for the Third International Mathematics and Science study for four countries and two languages (3,000 to 11,000 cases in each comparison group) reveal amounts and sources of DIF. (SLD)

International comparative educational studies make use of test instruments originally developed in English by international panels of experts, but that are ultimately administered in the language of instruction of the students. The comparability of the different language versions of these assessments is a critical issue in validating the…

Discusses how responses to multiple-choice items could be interpreted, demonstrates how responses on constructed-response items can be analyzed, and examines interactions between item characteristics and student responses. Uses information, specifically items and student responses, from the Third International Mathematics and Science Study…

Examined the performance of Irish students on multiple-choice, short-answer, and extended-response item sets from the Third International Mathematics and Science Study to determine whether Ireland's relative rank among the more than 40 countries involved remained stable. Findings provide additional evidence that comparing student achievement…

Examined whether changes in format of mathematics items in the Third International Mathematics and Science Study (TIMSS) had any effect on gender differences in performance using a Swedish sample of 8,851 sixth, seventh, and eighth graders. Results show no significant changes in gender differences when item format is altered. (SLD)

Data from the Third International Mathematics and Science Study (TIMSS) were examined to determine the extent to which the rank ordering of countries based on pupil test performance was consistent across three different item formats: multiple-choice, short-answer, and extended-response. Findings from the analysis are used to make the case that…