ERIC Number: ED596610
Record Type: Non-Journal
Publication Date: 2017-Jun
Pages: 8
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Towards Reliable and Valid Measurement of Individualized Student Parameters
Liu, Ran; Koedinger, Kenneth R. K
International Educational Data Mining Society, Paper presented at the International Conference on Educational Data Mining (EDM) (10th, Wuhan, China, Jun 25-28, 2017)
Research in Educational Data Mining could benefit from greater efforts to ensure that models yield reliable, valid, and interpretable parameter estimates. These efforts have especially been lacking for individualized student-parameter models. We collected two datasets from a sizable student population with excellent "depth" -- that is, many observations for each skill for each student. We fit two models, the Individualized-slope Additive Factors Model (iAFM) and Individualized Bayesian Knowledge Tracing (iBKT), both of which individualize for student ability and student learning rate. Estimates of student ability were reliable and valid: they were consistent across both models and across both datasets, and they significantly predicted out-of-tutor pretest data. In one of the datasets, estimates of student learning rate were reliable and valid: consistent across models and significantly predictive of pretest-posttest gains. This is the first demonstration that statistical models of data resulting from students' use of learning technology can produce reliable and valid estimates of individual student learning rates. Further, we sought to interpret and understand what differentiates a student with a high estimated learning rate from a student with a low one. We found that learning rate is significantly related to estimates of student ability (prior knowledge) and self-reported measures of diligence. Finally, we suggest a variety of possible applications of models with reliable estimates of individualized student parameters, including a more novel, straightforward way of identifying wheel spinning. [For the full proceedings, see ED596512.]
Descriptors: Data Analysis, Intelligent Tutoring Systems, Bayesian Statistics, Pretests Posttests, Educational Technology, Models, Reliability, Validity, Prior Learning, Goodness of Fit, High School Students, Geometry, Mathematics Instruction, Learning Processes, Identification, Mastery Learning, Individual Differences, Skill Development, Equations (Mathematics), Honors Curriculum, Inclusion, Mathematics Skills, General Education, Comparative Analysis
International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: High Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED); National Science Foundation (NSF)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305B110003; SBE0836012
Author Affiliations: N/A