ERIC Number: EJ1196127
Record Type: Journal
Publication Date: 2018
Pages: 10
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-1531-7714
EISSN: N/A
Available Date: N/A
Fairness Concerns of Discrete Option Multiple Choice Items
Eckerly, Carol; Smith, Russell; Sowles, John
Practical Assessment, Research & Evaluation, v23 n16 Nov 2018
The Discrete Option Multiple Choice (DOMC) item format was introduced by Foster and Miller (2009) with the intent of improving the security of test content. However, by changing the amount and order of the content presented, the test taking experience varies by test taker, thereby introducing potential fairness issues. In this paper we investigated fairness concerns by evaluating the impact on test takers of the differing testing experiences when items are administered in the DOMC format. Specifically, we described the impact of the presentation order of the key on item difficulty and discrimination as well as the cumulative impact at the test level. We recommend not including DOMC items in exams until the methodology of scoring test takers on these items is revised to address specific fairness concerns identified in this paper.
Descriptors: Culture Fair Tests, Multiple Choice Tests, Testing, Test Items, Test Format, Scoring, Sample Size, Difficulty Level, Reaction Time, Scores, Computer Assisted Testing, Computer Security
Center for Educational Assessment. 813 North Pleasant Street, Amherst, MA 01002. e-mail: pare@umass.edu; Tel: 413-577-2180; Web site: https://scholarworks.umass.edu/pare
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A