NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED615500
Record Type: Non-Journal
Publication Date: 2021
Pages: 6
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
A Novel Algorithm for Aggregating Crowdsourced Opinions
Prihar, Ethan; Heffernan, Neil
International Educational Data Mining Society, Paper presented at the International Conference on Educational Data Mining (EDM) (14th, Online, Jun 29-Jul 2, 2021)
Similar content has tremendous utility in classroom and online learning environments. For example, similar content can be used to combat cheating, track students' learning over time, and model students' latent knowledge. These different use cases for similar content all rely on different notions of similarity, which make it difficult to determine contents' similarities. Crowdsourcing is an effective way to identify similar content in a variety of situations by providing workers with guidelines on how to identify similar content for a particular use case. However, crowdsourced opinions are rarely homogeneous and therefore must be aggregated into what is most likely the truth. This work presents the Dynamically Weighted Majority Vote method. A novel algorithm that combines aggregating workers' crowdsourced opinions with estimating the reliability of each worker. This method was compared to the traditional majority vote method in both a simulation study and an empirical study, in which opinions on seventh grade mathematics problems' similarity were crowdsourced from middle school math teachers and college students. In both the simulation and the empirical study the Dynamically Weighted Majority Vote method outperformed the traditional majority vote method, suggesting that this method should be used instead of majority vote in future crowdsourcing endeavors. [For the full proceedings, see ED615472.]
International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: Junior High Schools; Middle Schools; Secondary Education; Higher Education; Postsecondary Education; Elementary Education; Grade 7
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF); Institute of Education Sciences (ED); Office of Postsecondary Education (ED); Office of Elementary and Secondary Education (ED); Office of Naval Research (ONR) (DOD)
Authoring Institution: N/A
Grant or Contract Numbers: 1917808; 1931523; 1940236; 1917713; 1903304; 1822830; 1759229; 1724889; 1636782; 1535428; 1440753; 1316736; 1252297; 1109483; DRL1031398; R305A170137; R305A170243; R305A180401; R305A120125; R305C100024; P200A180088; P200A150306; N000141812768; R305A170641
Author Affiliations: N/A