In this paper we assert that the practice of principled assessment design renders traditional standard-setting methodology redundant at best and contradictory at worst. We describe the rationale for, and methodological details of, Embedded Standard Setting (ESS; previously, Engineered Cut Scores. Lewis, 2016), an approach to establish performance…

Setting performance standards is a judgmental process involving human opinions and values as well as technical and empirical considerations. Although all cut score decisions are by nature somewhat arbitrary, they should not be capricious. Judges selected for standard-setting panels should have the proper qualifications to make the judgments asked…

These are the appendices for the technical report, "An Investigation of the Comparability of Commission-Approved Teaching Performance Assessment Models." California's Commission on Teacher Credentialing (Commission) requires all programs of preliminary multiple and single subject teacher preparation to use a Commission-approved Teaching…

Large-scale performance assessment programs are a longstanding reform tool. However, standard setting can be a challenge for assessment programs that use primarily non-standardized assessments. The purpose of this paper is to extend this field of research by explaining the standard setting methodology applied to one more recent instantiation of a…

This research aims to create and validate the quality of performance assessment using functional analysis. The researchers employed a design-based research method to create core competencies and their workload-based key outcome indicators as a preliminary study encompassing two phases, before formulating a standards-setting appraisal model to…

California's Commission on Teacher Credentialing (Commission) requires all programs of preliminary multiple and single subject teacher preparation to use a Commission-approved Teaching Performance Assessment (TPA) as one of the program completion requirements for prospective teacher candidates. Three TPA models were approved by the Commission: (1)…

In this paper we describe content focused, practical procedures to establish benchmarked performance standards for educational and other tests. To implement our procedures, we identify one or more external benchmark (e.g., NAEP or PISA performance levels, college and career readiness benchmarks); link to the external benchmarks to locate those…

Most states have developed or revised school accountability systems in response to the Every Student Succeeds Act (ESSA). While these systems include multiple indicators, to many stakeholders the outcome of central interest is the overall rating or classification produced for each school. These ratings are often used to identify schools that merit…

Within initial teacher education there is increasing pressure to enhance the use of assessment data to support students to improve their knowledge and skills, and to determine what standards they meet upon graduation. For such data to be useful, both programme designers and students require meaningful and comprehensive assessment reports on…