This paper introduces an application of cross-classified item response theory (IRT) modeling to an assessment utilizing the embedded standard setting (ESS) method (Lewis & Cook). The cross-classified IRT model is used to treat both item and person effects as random, where the item effects are regressed on the target performance levels (target…

Embedded Standard Setting (ESS; Lewis & Cook, 2020) transforms standard setting from a standalone workshop to an active part of the assessment development lifecycle. ESS purports to lower costs by eliminating the standard-setting workshop and enhance the validity argument by maintaining a consistent focus on the evidentiary relationship…

This study evaluated the efficacy of two proposed methods in an operational standard-setting study conducted for a high-stakes language proficiency test of the U.S. government. The goal was to seek low-cost modifications to the existing Yes/No Angoff method to increase the validity and reliability of the recommended cut scores using a convergent…

Setting cut scores on (MSTs) is difficult, particularly when the test spans several grade levels, and the selection of items from MST panels must reflect the operational test specifications. In this study, we describe, illustrate, and evaluate three methods for mapping panelists' Angoff ratings into cut scores on the scale underlying an MST. The…

In this study, the cut-off scores obtained from the Angoff, Angoff Y/N, Nedelsky and Ebel standard methods were compared with the 50 T score and the current cut-off score in various aspects. Data were collected from 448 students who took Module B1+ English Exit Exam IV and 14 experts. It was seen that while the Nedelsky method gave the lowest…

Given the significant global influence of the Common European Framework of Reference for Languages: Teaching, Learning, and Assessment (CEFR) on English language education, this study deals with aligning a university's academic reading tests to the CEFR. It aimed at validating the test construct of the academic reading tests in relation to the…

The National Assessment Governing Board (the Board) contracted with Pearson to design and implement a review of the achievement level descriptions (ALDs) for National Assessment of Educational Progress (NAEP) Grade 8 assessments in Science, U.S. History, and Civics. This document describes the procedural and technical aspects and outcomes of the…

The Nebraska Student-Centered Assessment System (NSCAS) is a statewide assessment system that embodies Nebraska's holistic view of students and helps them prepare for success in postsecondary education, career, and civic life. It uses multiple measures throughout the year to provide educators and decision-makers at all levels with the insights…

The National Assessment Governing Board has a legislatively mandated responsibility to develop National Assessment of Educational Progress (NAEP) achievement levels. The Board Policy Statement on Developing Student Achievement Levels for the National Assessment of Educational Progress provides policy definitions of "NAEP Basic,"…