RobotReviewer is a tool for automatically assessing the risk of bias in randomized controlled trials, but there is limited evidence of its reliability. We evaluated the agreement between RobotReviewer and humans regarding the risk of bias assessment based on 1955 randomized controlled trials. The risk of bias in these trials was assessed via two…

In multisite trials, learning about treatment effect variation across sites is critical for understanding where and for whom a program works. Unadjusted comparisons, however, capture "compositional" differences in the distributions of unit-level features as well as "contextual" differences in site-level features, including…

In multisite trials, learning about treatment effect variation across sites is critical for understanding where and for whom a program works. Unadjusted comparisons, however, capture "compositional" differences in the distributions of unit-level features as well as "contextual" differences in site-level features, including…

Indirect comparisons are used to obtain estimates of relative effectiveness between two treatments that have not been compared in the same randomized controlled trial, but have instead been compared against a common comparator in separate trials. Standard indirect comparisons use only aggregate data, under the assumption that there are no…

Given the widespread use of non-experimental (NE) methods for assessing program impacts, there is a strong need to know whether NE approaches yield causally valid results in field settings. In within-study comparison (WSC) designs, the researcher compares treatment effects from an NE with those obtained from a randomized experiment that shares the…

Since the 1990s, there have been repeated calls for the systematic use of randomised controlled trials (RCTs) to inform educational decision-making. The advent of the Education Endowment Foundation (EEF) -- described as England's What Works Centre for Education -- in 2011 has made this a reality in England: by 2020, over a third of English schools…

Since its inception in the 1980s, the meta-ethnography approach for synthesising qualitative study accounts has been used extensively in health and social care research and to a lesser extent in educational research. The aim of this article is to reflect on the evolution of the method in both fields. It starts by describing the meta-ethnography…

In this paper we evaluate the effectiveness of an adaptive online learning platform, designed to support Chinese students in learning the English language. The adaptive platform is studied in three studies, where the experimental platform is compared to an alternate, non-adaptive platform, with random assignment to conditions (the adaptive…

Most evaluators have embraced the goal of evidence-based practice (EBP). Yet, many have criticized EBP review systems that prioritize randomized control trials and use various criteria to limit the studies examined. They suggest this could produce policy recommendations based on small, unrepresentative segments of the literature and recommend a…

"TNTP Teaching Fellows" is a highly selective route to teacher certification that aims to prepare people to teach in high-need public schools. The program recruits professionals seeking to change careers and recent college graduates who are not certified teachers. "TNTP Teaching Fellows" expects its participants to teach for…

The terms applicability, generalizability, external validity and transferability are related, sometimes used interchangeably and have in common that they lack a clear and consistent definition in the classic epidemiological literature. However, all of these terms generally describe one overarching theme: whether or not available research evidence…

Improving academic, behavioural, and social outcomes of students through empirical research has been a firm commitment among researchers, policy-makers, and other professionals in education across Europe and the United States (U.S.). To assist in building scientific evidences, executive bodies such as the European Commission and the Institute for…

Experimental evaluations that involve the educational system usually involve a hierarchical structure (students are nested within classrooms that are nested within schools, etc.). Concerns about contamination, where research subjects receive certain features of an intervention intended for subjects in a different experimental group, have often led…

The Mathematics Mastery programme is a whole-school approach to teaching mathematics that aims to raise attainment for all pupils and close the attainment gap between pupils from low income families and their peers. The programme aims to deepen pupils' conceptual understanding of key mathematical concepts. This summary is based on two randomised…

Use of multivariate analysis (e.g., multivariate analysis of variance) is common when normally distributed outcomes are collected in intervention research. However, when mixed responses--a set of normal and binary outcomes--are collected, standard multivariate analyses are no longer suitable. While mixed responses are often obtained in…