Conducting high-quality overviews of reviews (OoR) is time-consuming. Because the quality of systematic reviews (SRs) varies, it is necessary to critically appraise SRs when conducting an OoR. A well-established appraisal tool is A Measurement Tool to Assess Systematic Reviews (AMSTAR) 2, which takes about 15-32 min per application. To save time,…

Using R to conduct univariate meta-analyses is becoming common for publication. However, R can also conduct multivariate meta-analysis (MMA). However, newcomers to both R and MMA may find using R to conduct MMA daunting. Given that, R may not be easy for those unfamiliar with coding. Likewise, MMA is a topic of advanced statistics. Thus, it may be…

This article describes and justifies a reporting standard to improve data use documentation in Agent-Based Modelling. Following the development of reporting standards for models themselves, empirical modelling has now developed to the point where these standards need to take equally effective account of data use (which previously has tended to be…

Despite best efforts, the student experience remains poorly understood. One under-explored approach to understanding the student experience is the use of big data analytics. The reported study is a work in progress aimed at exploring the value of big data methods for understanding the student experience. A big data analysis of an open dataset of…

The purpose of this study was to develop and measure the usability of the software prototype of formative assessment in the LMS. This study was applied in terms of research objective and mixed method (qualitative-quantitative) in terms of data collection in which an exploratory sequential mixed methods design was used. In addition, in order to…

In the last few years, Qualitative Comparative Analysis (QCA) has become one of the most important data analysis methods in comparative research. According to the guidelines of this method, there are certain steps that a researcher needs to follow, before causally analyzing the data for necessary and sufficient conditions. One of these steps is…

Effect sizes are important because they are an accessible way to indicate the practical importance of observed associations or differences. Standardized mean difference (SMD) effect sizes, such as Cohen's d, are widely used in education and the social sciences -- in part because they are relatively easy to calculate. However, SMD effect sizes…

This dissertation advances research on evaluation (RoE) through a trio of studies focusing on the role of context and the innovative use of Linguistic Inquiry and Word Count (LIWC) software in formative evaluation in a qualitative research project. The first article extends Coryn et al. (2017) and elucidates how various contextual…

Purpose: The purpose of this dissertation is to investigate the feasibility of using tests of robustness in peer review. This study involved selecting three high-impact papers which featured open data and utilized bioinformatic analyses but provided no source code and refactoring these to allow external survey participants to swap tools,…

To meet student demand for responsive, adaptable, and up-to-date online courses, educators and learning designers need tools to analyse student interactions with their peers, educators and learning resources. Learning Management Systems (LMSs) store large volumes of detailed user data, but offer only limited, pre-set reports and visualizations to…

We examined how Learning Analytics literature represents participants from diverse societies by comparing the studies published with samples from WEIRD (Western, Industrialized, Rich, Democratic) nations versus non-WEIRD nations. By analyzing the Learning Analytics studies published during 2015-2019 (N = 360), we found that most of the studies…

Missing data are exceedingly common across a variety of disciplines, such as educational, social, and behavioral science areas. Missing not at random (MNAR) mechanism where missingness is related to unobserved data is widespread in real data and has detrimental consequence. However, the existing MNAR-based methods have potential problems such as…

The purpose of this paper is to introduce the Five-Level Qualitative Data Analysis (5LQDA) method for ATLAS.ti as a way to intentionally design methodological approaches applicable to the field of evaluation. To demonstrate my analytical process using ATLAS.ti, I use examples from an existing evaluation of a STEM Peer Learning Assistant program.…

There has been an increasing demand for healthcare analytics skills and competence by healthcare organizations. Although many universities have established programs and courses on healthcare analytics, most of these curricula have been designed for information systems (IS), information technology (IT), or analytics students. It is unclear how…

Tools for automatic grading programming assignments, also known as Online Judges, have been widely used to support computer science (CS) courses. Nevertheless, few studies have used these tools to acquire and analyse interaction data to better understand the students' performance and behaviours, often due to data availability or inadequate…