In this paper we assert that the practice of principled assessment design renders traditional standard-setting methodology redundant at best and contradictory at worst. We describe the rationale for, and methodological details of, Embedded Standard Setting (ESS; previously, Engineered Cut Scores. Lewis, 2016), an approach to establish performance…

Setting performance standards is a judgmental process involving human opinions and values as well as technical and empirical considerations. Although all cut score decisions are by nature somewhat arbitrary, they should not be capricious. Judges selected for standard-setting panels should have the proper qualifications to make the judgments asked…

Large-scale performance assessment programs are a longstanding reform tool. However, standard setting can be a challenge for assessment programs that use primarily non-standardized assessments. The purpose of this paper is to extend this field of research by explaining the standard setting methodology applied to one more recent instantiation of a…

This research aims to create and validate the quality of performance assessment using functional analysis. The researchers employed a design-based research method to create core competencies and their workload-based key outcome indicators as a preliminary study encompassing two phases, before formulating a standards-setting appraisal model to…

In this paper we describe content focused, practical procedures to establish benchmarked performance standards for educational and other tests. To implement our procedures, we identify one or more external benchmark (e.g., NAEP or PISA performance levels, college and career readiness benchmarks); link to the external benchmarks to locate those…

Within initial teacher education there is increasing pressure to enhance the use of assessment data to support students to improve their knowledge and skills, and to determine what standards they meet upon graduation. For such data to be useful, both programme designers and students require meaningful and comprehensive assessment reports on…

A task force was created in a small, AACSB-accredited business school to develop a more comprehensive set of standards for faculty performance. The task force relied heavily on faculty input to identify and describe key dimensions that capture effective teaching and service performance. The result is a multi-dimensional framework that will be used…

Education has a long-standing practice of turning worthwhile learning goals into lists of bits. One might even say that this practice is the original sin in curriculum design: take a complex whole, divide it into small pieces, string those together in a rigid sequence of instruction and testing, and call completion of this sequence…

Given the rising burden of healthcare costs, both patients and healthcare purchasers are interested in discerning which physicians deliver quality care. We proposed a methodology to assess physician clinical performance in preventive cardiology care, and determined a benchmark for minimally acceptable performance. We used data on eight…

This is an exciting time to be in science education. New science standards are being developed by a group of science educators from across the country, working with 26 states in a process managed by Achieve, Inc., a non-profit education reform organization. The development of the "Next Generation Science Standards" (NGSS) promises to be the most…

This paper aims to compare various Performance Management Systems (PMS) for business school in order to find the strengths of each standard as inputs to design new model of PMS. There are many critical aspects and gaps notified for new model to improve performance and even recognized that self evaluation performance management is not well…

This paper explores the views of nursing and medical domain experts in considering the standards for a specific-purpose English language screening test, the Occupational English Test (OET), for professional registration for immigrant health professionals. Since individuals who score performances in the test setting are often language experts…

The purpose of this study is to describe a new approach for evaluating the judgments of standard-setting panelists within the context of the bookmark procedure. The bookmark procedure is widely used for setting performance standards on high-stakes assessments. A many-faceted Rasch (MFR) model is proposed for evaluating the bookmark judgments of…

In this paper, we report on a study to quantify the impact on student learning and on student assessment literacy of a brief assessment literacy intervention. We first define "assessment literacy" then report on the development and validation of an assessment literacy measurement instrument. Using a pseudo-experimental design, we…

Literature shows that feedback that is specific, immediate and goal-oriented is effective on (pre-service) teachers' performance. Synchronous coaching gives this kind of feedback. Due to immediateness of feedback, pre-service teachers can suffer from cognitive load. We propose a set of standardised keywords through which this performance feedback…