NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED616523
Record Type: Non-Journal
Publication Date: 2017
Pages: 22
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-1929-7750
EISSN: N/A
Available Date: N/A
How Flexible Is Your Data? A Comparative Analysis of Scoring Methodologies across Learning Platforms in the Context of Group Differentiation
Ostrow, Korinn S.; Wang, Yan; Heffernan, Neil T.
Grantee Submission, Journal of Learning Analytics v4 n2 p91-112 2017
Data is flexible in that it is molded by not only the features and variables available to a researcher for analysis and interpretation, but also by how those features and variables are recorded and processed prior to evaluation. "Big Data" from online learning platforms and intelligent tutoring systems is no different. The work presented herein questions the quality and flexibility of data from two popular learning platforms, comparing binary measures of problem-level accuracy, the scoring method typically used to inform learner analytics, with partial credit scoring, a more robust, real-world methodology. This work extends previous research by examining how the manipulation of scoring methodology has the potential to alter outcomes when testing hypotheses, or specifically, when looking for significant differences between groups of students. Datasets from ASSISTments and Cognitive Tutor are used to assess the implications of data availability and manipulation within twelve mathematics skills. A resampling approach is used to determine the size of equivalent samples of high- and low-performing students required to reliably differentiate performance when considering each scoring methodology. Results suggest that in eleven out of twelve observed skills, partial credit offers more efficient group differentiation, increasing analytic power and reducing Type II error. Alternative applications of this approach and implications for the Learning Analytics community are discussed. [This article was published in "Journal of Learning Analytics" (EJ1149168).]
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF); Institute of Education Sciences (ED); Office of Postsecondary Education (ED), Higher Education Programs; Office of Naval Research (ONR) (DOD)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: ACI1440753; DRL1252297; DRL1109483; DRL1316736; DRL1031398; R305A120125; R305C100024
Author Affiliations: N/A