ERIC Number: ED599196
Record Type: Non-Journal
Publication Date: 2019-Jul
Pages: 6
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Do Learners Know What's Good for Them? Crowdsourcing Subjective Ratings of OERs to Predict Learning Gains
Whitehill, Jacob; Aguerrebere, Cecilia; Hylak, Benjamin
International Educational Data Mining Society, Paper presented at the International Conference on Educational Data Mining (EDM) (12th, Montreal, Canada, Jul 2-5, 2019)
We explored how learners' "subjective ratings" of open educational resources (OERs) in terms of how much they find them "helpful" can predict the actual "learning gains" associated with those resources as measured with pre- and post-tests. To this end, we developed a probabilistic model called GRAM (Gaussian Rating Aggregation Model) that combines subjective ratings from multiple learners into an aggregate quality score of each resource. Based on an experiment we conducted on Mechanical Turk (n = 304 participants with m = 17 math tutorial videos as resources), we found that aggregated subjective ratings are highly (and stat. sig.) predictive of the resources' average learning gains, with Pearson correlation of 0.78. Moreover, when predicting average learning gains of "new" learners, subjective scores were still predictive (Pearson correlation of 0.49) and attained higher prediction accuracy than a model that directly uses pre- and post-test data to estimate learning gains for each resource. These results have potential implications for large-scale learning platforms (e.g., MOOCs, Khan Academy) that assign resources (tutorials, explanations, hints, etc.) to learners based on the expected learning gains. [For the full proceedings, see ED599096.]
Descriptors: Open Educational Resources, Electronic Publishing, Collaborative Writing, Web Sites, Student Attitudes, Achievement Gains, Problem Solving, Inferences, Educational Quality
International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Office of Naval Research (ONR)
Authoring Institution: N/A
Grant or Contract Numbers: N000141812768
Author Affiliations: N/A

Peer reviewed
