NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED630861
Record Type: Non-Journal
Publication Date: 2023
Pages: 11
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
How to Open Science: Debugging Reproducibility within the Educational Data Mining Conference
Haim, Aaron; Gyurcsan, Robert; Baxter, Chris; Shaw, Stacy T.; Heffernan, Neil T.
International Educational Data Mining Society, Paper presented at the International Conference on Educational Data Mining (EDM) (16th, Bengaluru, India, Jul 11-14, 2023)
Despite increased efforts to assess the adoption rates of open science and robustness of reproducibility in sub-disciplines of education technology, there is a lack of understanding of why some research is not reproducible. Prior work has taken the first step toward assessing reproducibility of research, but has assumed certain constraints which hinder its discovery. Thus, the purpose of this study was to replicate previous work on papers within the proceedings of the "International Conference on Educational Data Mining" to accurately report on which papers are reproducible and why. Specifically, we examined 208 papers, attempted to reproduce them, documented reasons for reproducibility failures, and asked authors to provide additional information needed to reproduce their study. Our results showed that out of 12 papers that were potentially reproducible, only one successfully reproduced all analyses, and another two reproduced most of the analyses. The most common failure for reproducibility was failure to mention libraries needed, followed by non-seeded randomness. [For the complete proceedings, see ED630829. Additional funding for this paper was provided by the U.S. Department of Education's Graduate Assistance in Areas of National Need (GAANN).]
International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
Publication Type: Speeches/Meeting Papers; Information Analyses
Education Level: N/A
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF); Institute of Education Sciences (ED); Office of Elementary and Secondary Education (OESE) (ED), Education Innovation and Research (EIR); Office of Naval Research (ONR) (DOD); Federal Highway Administration (FHWA), National Highway Institute (NHI)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: 2118725; 2118904; 1950683; 1917808; 1931523; 1940236; 1917713; 1903304; 1822830; 1759229; 1724889; 1636782; 1535428; R305N210049; R305A170137; R305A170243; R305A180401; R305A120125; R305R220012; P200A180088; P200A150306; U411B190024; S411B210024; N000141812768; R44GM146483; R305D210031
Author Affiliations: N/A