The aim of this study was to compare, by simulation, the accuracy of mapping a cut-score from one test to another by expert judgement (using the Angoff method) versus the accuracy with a small-sample equating method (chained linear equating). As expected, the standard-setting method resulted in more accurate equating when we assumed a higher level…

Childhood trauma is an all-too common factor in the lives of students and their families. Schools and communities across America are more likely to serve families that have experienced trauma (Anderson, 2016). Although trauma or toxic episodes can impact families across the economic spectrum, children living in poverty, in socially isolated areas,…

Assessment and evaluation at all levels of educational systems have become policy priorities for many countries. Two common reasons for this are student learning expectations and accountability. Although much effort has been put into the creation and refinement of content standards, standardized tests, and methods for using testing results, there…

The New Zealand Government has announced a change package in response to a recent review of the National Certificates of Educational Achievement (NCEA). In this article, Charles Darr, a chief researcher at the New Zealand Council for Educational Research, outlines several elements of standards-based assessment that can usefully inform NCEA's…

Setting performance standards is a judgmental process involving human opinions and values as well as technical and empirical considerations. Although all cut score decisions are by nature somewhat arbitrary, they should not be capricious. Judges selected for standard-setting panels should have the proper qualifications to make the judgments asked…

Subscore reporting under item response theory models has always been a challenge partly because the test length of each subdomain is limited for precisely locating individuals on multiple continua. Diagnostic classification models (DCMs), providing a pass/fail decision and associated probability of pass on each subdomain, are promising…

An important consideration in standard setting is recruiting a group of panelists with different experiences and backgrounds to serve on the standard-setting panel. This study uses data from 14 different Angoff standard settings from a variety of medical imaging credentialing programs to examine whether people with different professional roles and…

In this study, we present the development of individualized feedback for a large-scale listening assessment by combining standard setting and cognitive diagnostic assessment (CDA) approaches. We used the performance data from 3,358 students' item-level responses to a field test of a national EFL test primarily intended for tertiary-level EFL…

These are the appendices for the technical report, "An Investigation of the Comparability of Commission-Approved Teaching Performance Assessment Models." California's Commission on Teacher Credentialing (Commission) requires all programs of preliminary multiple and single subject teacher preparation to use a Commission-approved Teaching…

This paper investigates what matters to medical domain experts when setting standards on a language for specific purposes (LSP) English proficiency test: the Occupational English Test's (OET) writing sub-test. The study explores what standard-setting participants value when making performance judgements about test candidates' writing responses,…

Seldom have comparative studies of educational assessment systems been undertaken, especially regarding their standard setting procedures. This study examines the effects of governance structures on the power relations in standard setting in the dominant school-leaving or university-entrance examinations. We present acritical analysis of the…

This brief is part of a four-part series examining the past, present, and future of modern school accountability systems. With the dual forces of the COVID-19 pandemic and the national call to action on racial inequity, the question of how we should measure and hold schools accountable for the impact they have on students is more urgent than ever.…

New Meridian Corporation has developed the "Quality Testing Standards and Criteria for Comparability Claims" (QTS). The goal of the QTS is to provide guidance to states that are interested in including content from the New Meridian item bank and intend to make comparability claims with "other assessments" that include New…

New Meridian Corporation has developed the "Quality Testing Standards and Criteria for Comparability Claims" (QTS). The goal of the QTS is to provide guidance to states that are interested in including content from the New Meridian item bank and intend to make comparability claims with "other assessments" that include New…

A significant gap has been identified in research-based findings and classroom instructional practices of hearing ASL teachers in high schools in the U.S. (Rosen, 2015). Research has shown the absence of standardized ASL instructional guidelines, rules, and procedures has created a significant gap in teaching and learning practices at the high…