This study aims to design mathematical literacy instruments that have evidence of content and construct validity and are reliable for use as an assessment for learning. The research involved eight experts as instrument validators and 273 eighth-grade students of junior high school in Yogyakarta Province. The results showed that the ten…

The aim of this study is to construct a STEM competencies assessment framework and provide validity evidence by empirically testing its structure. Common interdisciplinary assessment frameworks for STEM seem to be scarce in the literature. Many studies use students' mathematics or science scores obtained from large-scale assessments or exams to…

The aim of this research is to develop a measurement instrument that is valid and reliable, called the Middle School-Life Science Concept Inventory (MS-LSCI), for the purpose of measuring the life science conceptual understanding of middle school-level students. Although there are several existing concept inventories related to biology concepts…

Large scale international assessments depend on invariance of measurement across countries. An important consideration when observing cross-national differential item functioning (DIF) is whether the DIF actually reflects a source of bias, or might instead be a methodological artifact reflecting item response theory (IRT) model misspecification.…

Effectively evaluating the credibility and accuracy of multiple sources is critical for college readiness. We developed 24 source evaluation tasks spanning four predicted difficulty levels of a hypothesized learning progression (LP) and piloted these tasks to evaluate the utility of an LP-based approach to designing formative literacy assessments.…

Wording effects associated with positively and negatively worded items have been found in many scales. Such effects may threaten construct validity and introduce systematic bias in the interpretation of results. A variety of models have been applied to address wording effects, such as the correlated uniqueness model and the correlated traits and…

One of the most important dynamics of the educational setting all over the world is examinations, and some of those are English tests. In Turkey, English tests for the students preparing for the high schools were included in the national exams with the Level Determination Exam (SBS) in 2008 for the first time, and Transition Examination from…

The purpose of this study is to construct a digital culture test in light of the Item Response Theory and to investigate its psychometric properties. The study sample consisted of six hundred fifty (650) male and female students in the eighth grade from the Directorate of Education and Teaching of Salt District. To obtain the results, the…

This study examined the existence of latent classes in TIMSS 2015 data from three countries, Singapure, Turkey and South Africa, were analyzed using Mixture Item Response Theory (MixIRT) models (Rasch, 1PL, 2PL and 3PL) on 18 multiple-choice items in the science subtest. Based on the findings, it was concluded that the data obtained from TIMSS…

The research shared in this conference paper report illustrates how an iterative process to item development that involves expert review and cognitive lab interviews with students can be used to collect evidence of validity for assessment items. Analysis of students' reasoning was also used to expand a model for identifying conceptions and…

The National Assessment of Educational Progress's (NAEP's) transition to an entirely digitally based assessment (DBA) began in 2017. As part of this transition, new types of NAEP items have begun to be developed that leverage the DBA environment to measure a wider range of knowledge and skills. These new item types include the science…

Integrating engineering into the K-12 science curriculum continues to be a focus in national reform efforts in science education. Although there is an increasing interest in research in and practice of integrating engineering in K-12 science education, to date only a few studies have focused on the development of an assessment tool to measure…

This paper provides the National Center for Education Statistics (NCES), National Assessment Governing Board (NAGB), and the National Assessment of Educational Progress (NAEP) community with information that may help maintain the validity and utility of the NAEP assessments for civics and U.S. history as revisions are planned to the NAEP…

Although there are several comprehensive measures of children's anxiety symptoms, many have limited utility for screening within school settings. As such, the primary purpose of this dissertation was to develop a brief self-report anxiety measure appropriate for use with students in Grades 3-8. An initial pool of 50 items was developed based on…

Validity and reliability are among the most crucial characteristics of a test. One of the steps to make sure that a test is valid and reliable is to examine the bias in test items. The purpose of this study was to examine the bias in 2012 Placement Test items in terms of gender variable using Rasch Model in Turkey. The sample of this study was…