The subjective aspect of standard-setting is often criticized, yet data-driven standard-setting methods are rarely applied. Therefore, we applied a mixture Rasch model approach to setting performance standards across several testing programs of various sizes and compared the results to existing passing standards derived from traditional…

The indefinite nature of test fairness and different interpretations and definitions of the concept have stirred a lot of controversy over the years, necessitating the reconceptualization of the concept. On this basis, this study aimed to explore the empirical validity of Kunnan's (2008) Test Fairness Framework (TFF) and revisit the established…

Evaluation decisions regarding students' success in Open Education faculties such as pass/fail based on cut-off scores affect the quality of these systems. The qualification of Open Education students to obtain a bachelor's or associate's degree is determined by their passing grade. The purpose of this study was to investigate whether the minimum…

Establishing a cut score is a crucial aspect of the test development process since the selected cut score has the potential to impact students' performance outcomes and shape instructional strategies within the classroom. Therefore, it is vital for those involved in test development to set a cut score that is both fair and justifiable. This cut…

An Angoff standard setting study generally yields judgments on a number of items by a number of judges (who may or may not be nested in panels). Variability associated with judges (and possibly panels) contributes error to the resulting cut score. The variability associated with items plays a more complicated role. To the extent that the mean item…

Setting performance standards is a judgmental process involving human opinions and values as well as technical and empirical considerations. Although all cut score decisions are by nature somewhat arbitrary, they should not be capricious. Judges selected for standard-setting panels should have the proper qualifications to make the judgments asked…

This paper investigates what matters to medical domain experts when setting standards on a language for specific purposes (LSP) English proficiency test: the Occupational English Test's (OET) writing sub-test. The study explores what standard-setting participants value when making performance judgements about test candidates' writing responses,…

Large-scale performance assessment programs are a longstanding reform tool. However, standard setting can be a challenge for assessment programs that use primarily non-standardized assessments. The purpose of this paper is to extend this field of research by explaining the standard setting methodology applied to one more recent instantiation of a…

Standard setting is the process of identifying the point(s) on a scale that serve to differentiate between individuals of distinct proficiency levels. While standard setting is ultimately a policy decision, most of the process is carried out by subject matter experts who are tasked with reconciling item-level or examinee-level information (e.g.…

Purpose: University placement test is an important admission policy priority in Nigeria, because it serves as a university-based selection criterion for placement of students into undergraduate programs in Nigeria. Although recently attention have been shifted on the call to develop a standard content and standardize the test, yet attention has…

We report on a standard-setting project in which the Item-Descriptor-Matching Method (IDM) and a complementary benchmarking approach were employed to align a suite of English language proficiency exams to the "Common European Framework of Reference" (CEFR), with a particular focus on the integrated and independent writing exams. Judges'…

The space between the two epigraphs presented at the beginning of this article marks the difference between an insider and an outsider. Ms Barbieri, from "Voices from the Middle," made the first comments as an expression of what reflective practice -- observing and critiquing video tapes of hers and others' teaching -- could mean to her…

The National Assessment Governing Board has a legislatively mandated responsibility to develop National Assessment of Educational Progress (NAEP) achievement levels. The Board Policy Statement on Developing Student Achievement Levels for the National Assessment of Educational Progress provides policy definitions of "NAEP Basic,"…

In the field of second and foreign language learning, the Common European Framework of Reference for languages (CEFR) is widely-used for setting language proficiency standards within European, and increasingly global, contexts. Few studies, however, have investigated the ways in which systemic, macro-level factors within national educational…

The Angoff standard setting method relies on content experts to review exam items and make judgments about the performance of the minimally proficient examinee. Unfortunately, at times content experts may have gaps in their understanding of specific exam content. These gaps are particularly likely to occur when the content domain is broad and/or…