Despite the rising popularity of big data, there is speculation that evaluators have been slow adopters of these new statistical approaches. Several possible reasons have been offered for why this is the case: ethical concerns, institutional capacity, and evaluator capacity and values. In this method note, we address one of these barriers and aim…

This study explored the core competencies, technological skills, functional proficiencies, and professional experiences of data scientists at higher education institutions. The specific population of interest was higher education administrators and staff professionals identified as data scientists. This study was informed by the following guiding…

This dissertation address two significant challenges in the causal inference workflow for Big Observational Data. The first is designing Big Observational Data with high-dimensional and heterogeneous covariates. The second is performing uncertainty quantification for estimates of causal estimands that are obtained from the application of black box…

Research based on educational data mining conducted at academic institutions is often limited by the institutional policy with regard to the type of learning management system and the detail level of its activity reports. Often, researchers deal with only raw data. Such data normally contain numerous fictitious user activities that can create a…

This qualitative case study sought to discover how school-level data teams can intentionally use effective data practices to identify and implement high-leverage interventions that support all students, including Black and Latinx boys, in attaining the necessary academic requirements for high school graduation. The researcher analyzed data from…

This essay describes the design, and subsequent redesign, of the research methods courses included in an Instructional and Performance Technology (IPT) EdD program at a regional comprehensive university in the southeast United States. The program under examination was developed based on the principles of the Carnegie Project for the Education…

This contribution deals with effect measures for covariates in ordinal data models to address the interpretation of the results on the extreme categories of the scales, evaluate possible response styles, and motivate collapsing of extreme categories. It provides a simpler interpretation of the influence of the covariates on the probability of the…

Statistical literacy has a large and important role in the teaching of statistics. Most mathematics and statistics courses are hierarchical, and the earlier material forms the foundation for later material. We construct a hierarchical structure for an introductory statistics course using Rasch analysis of the student scripts for the final…

In 2009, John Hattie's book Visible Learning: A Synthesis of over 800 Meta-Analyses Relating to Achievement brought big data to education. In the decade and a half since Visible Learning was originally published it has been aggressively marketed and has now grown into a large suite of branded books, tools, and products. Visible Learning continues…

The prediction of student performance and the analysis of students' learning behaviour play an important role in enhancing online courses. By analysing a massive amount of clickstream data that captures student behaviour, educators can gain valuable insights into the factors that influence students' academic outcomes and identify areas of…

Conducting high-quality overviews of reviews (OoR) is time-consuming. Because the quality of systematic reviews (SRs) varies, it is necessary to critically appraise SRs when conducting an OoR. A well-established appraisal tool is A Measurement Tool to Assess Systematic Reviews (AMSTAR) 2, which takes about 15-32 min per application. To save time,…

The objectives of this study were to identify gaps in educational training for undergraduate and graduate students in agricultural data science, propose paths for filling these gaps, and provide an annotated list of resources currently available to different training levels. Data in this study was collected through three voluntary surveys catered…

Managing and analysing large qualitative datasets pose a particular challenge for researchers seeking a consistent and rigorous approach to qualitative data analysis. This paper describes and demonstrates the development and adoption of a matrix tool to guide the qualitative data analysis of a large sample (N = 122) of interview data. The paper…

Although parallel analysis has been found to be an accurate method for determining the number of factors in many conditions with complete data, its application under missing data is limited. The existing literature recommends that, after using an appropriate multiple imputation method, researchers either apply parallel analysis to every imputed…

The present study used quantitative and qualitative measures to examine Iranian applied linguists' (mis-) conceptions of ethical issues in research. For this purpose, one hundred and twelve applied linguists completed a research ethics questionnaire constructed and validated by the researchers. In the follow-up qualitative phase, 15 applied…