The synthetic control method (SCM) is a popular approach for estimating the impact of a treatment on a single unit with panel data. Two challenges arise with higher frequency data (e.g., monthly versus yearly): (1) achieving excellent pre-treatment fit is typically more challenging; and (2) overfitting to noise is more likely. Aggregating data…

This paper argues for three aspects of tolerance with respect to QCA research: tolerance with respect to different approaches to QCA; producing QCA research with tolerance (work that is resistant to criticism); and for QCA researchers to be clear about the tolerance of the solutions they present -- especially in terms of calibration and truth…

In the last few years, Qualitative Comparative Analysis (QCA) has become one of the most important data analysis methods in comparative research. According to the guidelines of this method, there are certain steps that a researcher needs to follow, before causally analyzing the data for necessary and sufficient conditions. One of these steps is…

Diversity audits are frequently used as an assessment method to measure the diversity of a library collection. Yet, there is not frequent research on the aftermath of diversity audits, especially in the context of comparing data from several audits to assess the difference in the makeup of a library collection. In this article, the author…

Qualitative comparative analysis (QCA) is gaining ground in evaluation circles, but the number of applications is still limited. In this article, we consider the challenges that can emerge during a QCA evaluation by drawing on our experience of conducting one in the field of development cooperation. For each stage of the evaluation process, we…

A small subset of education studies analyzes school data collected seasonally (separating the summer from the school year). At first, this work was primarily known for documenting learning loss in the summers, but scholars have since recognized that observing how inequality changes between summer and school periods provides leverage for…

During drug development, a biomarker is sometimes identified as separating a patient population into those with more and those with less benefit from evaluated treatments. Consequently, later studies might be targeted, while earlier ones are performed in mixed patient populations. This poses a challenge in evidence synthesis, especially if only…

In the domain of programming, a growing number of algorithms automatically generate data-driven, next-step hints that suggest how students should edit their code to resolve errors and make progress. While these hints have the potential to improve learning if done well, few evaluations have directly assessed or compared the quality of different…

University rankings have become commonplace in higher education. Traditional quantified rankings do not merely measure educational performance: they equally grant status, enforce competition between institutions, and are emblematic for the ongoing capitalization of higher education. Drawing on the field of Science and Technology Studies, this…

Scores estimated from multidimensional item response theory (IRT) models are not necessarily comparable across dimensions. In this article, the concept of aligned dimensions is formalized in the context of Rasch models, and two methods are described--delta dimensional alignment (DDA) and logistic regression alignment (LRA)--to transform estimated…

Data science is a broad, interdisciplinary field concerned with the extraction of knowledge or insights from data, with the classification of data as a core, fundamental task. One of the most persistent challenges faced when performing classification is the class imbalance problem. Class imbalance refers to when the frequency with which each class…

Researchers in many disciplines work with ranking data. This data type is unique in that it is often deterministic in nature (the ranks of items "k"-1 determine the rank of item "k"), and the difference in a pair of rank scores separated by "k" units is equivalent regardless of the actual values of the two ranks in…

Recommender systems in educational contexts have proven effective to identify learning resources that fit the interests and needs of learners. Their usage has been of special interest in online self-learning scenarios to increase student retention and improve the learning experience. In current recommendation techniques, and in particular, in…

The explosion of Open Educational Resources (OERs) in the recent years creates the demand for scalable, automatic approaches to process and evaluate OERs, with the end goal of identifying and recommending the most suitable educational materials for learners. We focus on building models to find the characteristics and features involved in…

'School league tables' summarising the performance of secondary schools in England have been published annually since 1992. Although now a firmly established feature of the educational landscape, they have attracted criticism from those who point to their unintended negative consequences. These include 'teaching to the test', entering pupils for…