Assessments in higher education healthcare programmes can be challenging because they not only need to be fair, valid, and transparent, but it is also necessary to gauge safety, practical skill competence, and professionalism. One way to help maximise validity in practical assessments is to utilise 'standard setting' which aims to set a fair 'cut…

The indefinite nature of test fairness and different interpretations and definitions of the concept have stirred a lot of controversy over the years, necessitating the reconceptualization of the concept. On this basis, this study aimed to explore the empirical validity of Kunnan's (2008) Test Fairness Framework (TFF) and revisit the established…

Raters must score accurately and consistently for classroom observation scores to be valid. This requires (a) a standard defining when scoring is accurate and consistent enough and (b) measuring and remediating rater performance against that standard. Current practice has focused on this second problem to the exclusion of the first. My goal here…

The space between the two epigraphs presented at the beginning of this article marks the difference between an insider and an outsider. Ms Barbieri, from "Voices from the Middle," made the first comments as an expression of what reflective practice -- observing and critiquing video tapes of hers and others' teaching -- could mean to her…

This technical report documents the processes and procedures implemented to support the Spring 2021 Nebraska Student-Centered Assessment System (NSCAS) Phase I Pilot in English Language Arts (ELA), Mathematics, and Science assessments by NWEA® under the supervision of the Nebraska Department of Education (NDE). The technical report shows how the…

In Winter 2021-2022, the Nebraska Student-Centered Assessment System (NSCAS) assessments are administered in ELA and mathematics in Grades 3-8. In Spring 2021-2022, the NSCAS assessments are administered in English language arts (ELA) and mathematics in Grades 3-8 and in science in Grades 5 and 8. The purposes of the NSCAS assessments are to…

The Spring 2020 Nebraska Student-Centered Assessment System (NSCAS) General Summative testing was cancelled due to COVID-19. This technical report documents the processes and procedures that had been implemented to support the Spring 2020 assessments prior to the cancellation. The following sections are presented in this technical report: (1)…

In Fall and Winter 2022-2023, the NSCAS assessments were administered in ELA and mathematics for grades 3-8. In Spring 2022-2023, the NSCAS assessments were administered in English language arts (ELA) and mathematics for grades 3-8 and in science for grades 5 and 8. The purposes of the NSCAS assessments are to measure and report Nebraska students'…

Federal accountability requirements after the No Child Left Behind (NCLB) Act of 2001 and the need to report progress for various disaggregated subgroups of students meant that the methods used to set and articulate performance standards across the grades must be revisited. Several solutions that involve either "a priori" deliberations…

This technical report documents the processes and procedures implemented to support the Spring 2019 Nebraska Student-Centered Assessment System (NSCAS) General Summative English Language Arts (ELA), Mathematics, and Science assessments by NWEA® under the supervision of the Nebraska Department of Education (NDE). The technical report shows how the…

The consequences associated with the uses and interpretations of scores for many credentialing testing programs have important implications for a range of stakeholders. Within licensure settings specifically, results from examination programs are often one of the final steps in the process of assessing whether individuals will be allowed to enter…

This "Practitioner's Guide to Growth Models," commissioned by the Technical Issues in Large-Scale Assessment (TILSA) and Accountability Systems & Reporting (ASR), collaboratives of the "Council of Chief State School Officers," describes different ways to calculate student academic growth and to make judgments about the…

This study compares the efficacy of different strategies for translating item-level, proportion-correct standard-setting judgments into a theta-metric test cutoff score for use with item response theory (IRT) scoring, using Monte Carlo methods. Simulated Angoff-type ratings, consisting of 1,000 independent 75 Item x13 Rater matrices, were…

Background: To establish credible, defensible and acceptable passing scores for written tests is a challenge for health profession educators. Angoff procedures are often used to establish pass/fail decisions for written and performance tests. In an Angoff procedure judges' expertise and professional skills are assumed to influence their ratings of…

Only a small number of qualitative studies have investigated panelists' experiences during standard-setting activities or the thought processes associated with panelists' actions. This qualitative study involved an examination of the experiences of 11 panelists who participated in a prior, one-day standard-setting meeting in which either the…