ERIC Number: EJ1491384
Record Type: Journal
Publication Date: 2025-Dec
Pages: 31
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0022-0655
EISSN: EISSN-1745-3984
Available Date: 2025-09-09
The Precision and Bias of Cut Score Estimates from the Beuk Standard Setting Method
Joseph H. Grochowalski1; Lei Wan1; Lauren Molin1; Amy H. Hendrickson1
Journal of Educational Measurement, v62 n4 p687-717 2025
The Beuk standard setting method derives cut scores through expert judgment that balances content and normative perspectives. This study developed a method to estimate confidence intervals for Beuk settings and assessed their accuracy via simulations. Simulations varied SME panel size, expert agreement, cut score locations, score distributions, and decision alignment. Panels with 20+ participants provided precise and accurate cut score estimates if strongly agreed upon. Larger panels did not improve precision significantly. Cut score location influenced confidence interval widths, highlighting its importance in planning. Real data showed SME disagreement increased bias and variance of Beuk estimates. Use Beuk cut scores cautiously with small panels, flat score distributions, or significant expert disagreement.
Descriptors: Cutting Scores, Standard Setting, Accuracy, Statistical Bias, Simulation, Interrater Reliability
Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www-wiley-com.bibliotheek.ehb.be/en-us
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: 1The College Board

Peer reviewed
Direct link
