NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1381288
Record Type: Journal
Publication Date: 2023
Pages: 19
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0895-7347
EISSN: EISSN-1532-4818
Available Date: N/A
Dissecting Knowledge, Guessing, and Blunder in Multiple Choice Assessments
Abu-Ghazalah, Rashid M.; Dubins, David N.; Poon, Gregory M. K.
Applied Measurement in Education, v36 n1 p80-98 2023
Multiple choice results are inherently probabilistic outcomes, as correct responses reflect a combination of knowledge and guessing, while incorrect responses additionally reflect blunder, a confidently committed mistake. To objectively resolve knowledge from responses in an MC test structure, we evaluated probabilistic models that explicitly account for guessing, knowledge, and blunder using eight assessments (>9,000 responses) from an undergraduate biotechnology curriculum. A Bayesian implementation of the models, aimed at assessing their robustness to prior beliefs in examinee knowledge, showed that explicit estimators of knowledge are markedly sensitive to prior beliefs with scores as sole input. To overcome this limitation, we examined self-ranked confidence as a proxy knowledge indicator. For our test set, three levels of confidence resolved test performance. Responses rated as least confident were correct more frequently than expected from random selection, reflecting partial knowledge, but were balanced by blunder among the most confident responses. By translating evidence-based guessing and blunder rates to pass marks that statistically qualify a desired level of examinee knowledge, our approach finds practical utility in test analysis and design.
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF), Division of Molecular and Cellular Biosciences (MCB); National Heart, Lung, and Blood Institute (NHLBI) (DHHS/NIH)
Authoring Institution: N/A
Identifiers - Location: Canada
Grant or Contract Numbers: 2028902; HL155178
Author Affiliations: N/A