ERIC Number: EJ1430503
Record Type: Journal
Publication Date: 2024
Pages: 33
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-2157-2100
Available Date: N/A
The Knowledge Component Attribution Problem for Programming: Methods and Tradeoffs with Limited Labeled Data
Yang Shi; Robin Schmucker; Keith Tran; John Bacher; Kenneth Koedinger; Thomas Price; Min Chi; Tiffany Barnes
Journal of Educational Data Mining, v16 n1 p1-33 2024
Understanding students' learning of knowledge components (KCs) is an important educational data mining task and enables many educational applications. However, in the domain of computing education, where program exercises require students to practice many KCs simultaneously, it is a challenge to attribute their errors to specific KCs and, therefore, to model student knowledge of these KCs. In this paper, we define this task as the KC attribution problem. We first demonstrate a novel approach to addressing this task using deep neural networks and explore its performance in identifying expert-defined KCs (RQ1). Because the labeling process takes costly expert resources, we further evaluate the effectiveness of transfer learning for KC attribution, using more easily acquired labels, such as problem correctness (RQ2). Finally, because prior research indicates the incorporation of educational theory in deep learning models could potentially enhance model performance, we investigated how to incorporate learning curves in the model design and evaluated their performance (RQ3). Our results show that in a supervised learning scenario, we can use a deep learning model, code2vec, to attribute KCs with a relatively high performance (AUC > 75% in two of the three examined KCs). Further using transfer learning, we achieve reasonable performance on the task without any costly expert labeling. However, the incorporation of learning curves shows limited effectiveness in this task. Our research lays important groundwork for personalized feedback for students based on which KCs they applied correctly, as well as more interpretable and accurate student models.
Descriptors: Programming Languages, Undergraduate Students, Learning Processes, Teaching Models, Information Transfer, Data Collection, Data Use, Program Design, Cognitive Structures, Cognitive Processes
International Educational Data Mining. e-mail: jedm.editor@gmail.com; Web site: https://jedm.educationaldatamining.org/index.php/JEDM
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF)
Authoring Institution: N/A
Identifiers - Location: Virginia
Grant or Contract Numbers: 2013502; 2112635
Author Affiliations: N/A