Effectively evaluating the credibility and accuracy of multiple sources is critical for college readiness. We developed 24 source evaluation tasks spanning four predicted difficulty levels of a hypothesized learning progression (LP) and piloted these tasks to evaluate the utility of an LP-based approach to designing formative literacy assessments.…

There is a broad literature in multiple-choice test development, both in terms of item-writing guidelines, and psychometric functionality as a measurement tool. However, most of the published literature concerns multiple-choice testing in the context of expert-designed high-stakes standardized assessments, with little attention being paid to the…

The Neurodevelopmental Ecological Screening Tool (NEST) is a new instrument to screen children for developmental challenges. This article describes the validation of the NEST neurodevelopmental domain. Data were collected from a nationwide purposely restricted sample of caregivers of children aged 3-5 years (n = 231) living in poverty and…

University lecturers and coordinators of business and economics courses around the world are faced with the challenge that beginning students in these courses have heterogeneous entry conditions in terms of personal characteristics. This article focuses on the economic knowledge of German and Japanese beginning students in a business and economics…

In a signal detection theory (SDT) approach to multiple choice exams, examinees are viewed as choosing, for each item, the alternative that is perceived as being the most plausible, with perceived plausibility depending in part on whether or not an item is known. The SDT model is a process model and provides measures of item difficulty, item…

In Programme for International Student Assessment (PISA), item response theory (IRT) scaling is used to examine the psychometric properties of items and scales and to provide comparable test scores across participating countries and over time. To balance the comparability of IRT item parameter estimations across countries with the best possible…

The purpose of this paper is to briefly introduce two most common applications of multiple group item response theory (IRT) models, namely detecting differential item functioning (DIF) analysis and nonequivalent group score linking with a simultaneous calibration. We illustrate how to conduct those analyses using the "Stata" item…

This research examines the difficulty pattern of assertive communication scale instrument items containing spiritual values. The research and development design applies ADDIE work procedures (Analysis, Design, Development or Production, Implementation or delivery and Evaluation). The participants of the item development and item difficulty test…

The development and evaluation of valid assessments of scientific reasoning are an integral part of research in science education. In the present study, we used the linear logistic test model (LLTM) to analyze how item features related to text complexity and the presence of visual representations influence the overall item difficulty of an…

In 2017, the mathematics assessments that are part of the National Assessment of Educational Progress (NAEP) program underwent a transformation shifting the administration from paper-and-pencil formats to digitally-based assessments (DBA). This shift introduced new interactive item types that bring rich process data and tremendous opportunities to…

Curriculum-based measurement of oral reading fluency (CBM-R) is widely used across the United States as a strong indicator of comprehension and overall reading achievement, but has several limitations including errors in administration and large standard errors of measurement. The purpose of this study is to compare scoring methods and passage…

New MAP® Growth™ assessments are being developed that administer items more closely matched to the grade level of the student. However, MAP Growth items are calibrated with samples that typically consist of students from a variety of grades, including the target grade to which an item is aligned. While this choice of calibration sample is…

Formally describing and assessing the difficulty of learning and teaching material is important for quality assurance in university teaching, for aligning teaching and learning activities, and for easing communications among stakeholders such as teachers and students. This paper proposes a novel taxonomy to describe and quantify the difficulty…

To better understand Tennessee's new standardized science assessments, this quantitative study utilized a nonexperimental, descriptive-comparative design to compare the readability of the long-used science TCAP assessment with the newly created science TNReady assessment in grades three, four, and five. As new standards in the state boast higher…

The Chemistry Self-Concept Inventory (CSCI) is a widely used instrument within chemistry education research. Yet, agreement on its overall reliability and validity is lacking, and psychometric analyses of the instrument remain outstanding. This study examined the psychometric properties of the subscale and item function of the CSCI on 1,140 high…