Assessments in higher education healthcare programmes can be challenging because they not only need to be fair, valid, and transparent, but it is also necessary to gauge safety, practical skill competence, and professionalism. One way to help maximise validity in practical assessments is to utilise 'standard setting' which aims to set a fair 'cut…

This paper introduces an application of cross-classified item response theory (IRT) modeling to an assessment utilizing the embedded standard setting (ESS) method (Lewis & Cook). The cross-classified IRT model is used to treat both item and person effects as random, where the item effects are regressed on the target performance levels (target…

Professional associations support a field in many ways, including professionalization, setting standards of practice, advocating for members and the industry it supports, and generating and sharing knowledge, research, and scholarship. While they provide ongoing education, they can play a more intentional role in professional preparation. This…

Embedded Standard Setting (ESS; Lewis & Cook, 2020) transforms standard setting from a standalone workshop to an active part of the assessment development lifecycle. ESS purports to lower costs by eliminating the standard-setting workshop and enhance the validity argument by maintaining a consistent focus on the evidentiary relationship…

I consider recent attempts to establish standards, principles, and goals for artificial intelligence (AI) through the lens of educational measurement. Distinctions are made between generative AI and AI-adjacent methods and applications of AI in formative versus summative assessment contexts. While expressing optimism about its possibilities, I…

As technology becomes a more central support for students with disabilities in inclusive schools, there is a need to understand fidelity of implementation of technology-delivered interventions. We propose a framework for conceptualizing fidelity of the Goal Setting Challenge App, a technology-delivered, intervention based on the Self-Determined…

This study evaluated the efficacy of two proposed methods in an operational standard-setting study conducted for a high-stakes language proficiency test of the U.S. government. The goal was to seek low-cost modifications to the existing Yes/No Angoff method to increase the validity and reliability of the recommended cut scores using a convergent…

Unlike other aspects of Obstetrics and Gynecology (OBGYN) residency training, the research education curriculum and requirements are not standardized. It is unclear what constitutes core curricula in educational research programs, which components are most useful and efficacious, and how to use these findings to adapt existing programs. A…

There is continuing demand from practitioners, standard setters, and regulators for accountants to have the knowledge and competencies to contribute to sustainable business practices, reporting, and assurance. Educators at all levels of the accounting curricula have a responsibility to upskill. This research examines the extent of sustainability…

Effective communication skills are pivotal in health care, particularly when conveying distressing information to patients and their families. However, medical education still lacks the adoption of a universal model that can be incorporated into the curricula to train and assess students in effectively communicating with patients. This study aims…

The subjective aspect of standard-setting is often criticized, yet data-driven standard-setting methods are rarely applied. Therefore, we applied a mixture Rasch model approach to setting performance standards across several testing programs of various sizes and compared the results to existing passing standards derived from traditional…

Proposed is a new method of standard setting referred to as response vector for mastery (RVM) method. Under the RVM method, the task of panelists that participate in the standard setting process does not involve conceptualization of a borderline examinee and probability judgments as it is the case with the Angoff and bookmark methods. Also, the…

Performance standards have important consequences for all the stakeholders in the assessment of L2 academic writing. These standards not only describe the level of writing performance but also provide a basis for making evaluative decisions on the academic writing. Such a high-stakes role of the performance standards requires the enhancement of…

Despite the popularity of traditional fit index cutoffs like RMSEA [less than or equal to] 0.06 and CFI [greater than or equal to] 0.95, several studies have noted issues with overgeneralizing traditional cutoffs. Computational methods have been proposed to avoid overgeneralization by deriving cutoffs specifically tailored to the characteristics…

Setting cut scores on (MSTs) is difficult, particularly when the test spans several grade levels, and the selection of items from MST panels must reflect the operational test specifications. In this study, we describe, illustrate, and evaluate three methods for mapping panelists' Angoff ratings into cut scores on the scale underlying an MST. The…