In this article, we discuss the methodological implications of data and theory integration for Theory-Based Evaluation (TBE). TBE is a family of approaches to program evaluation that use program theories as instruments to answer questions about whether, how, and why a program works. Some of the groundwork about TBE has expressed the idea that a…

Qualitative interviews and focus groups are commonly used methods to elicit participants' voices in program evaluations. However, the use of these data-gathering methods can fall short of the goal; even with open-ended questions, the protocols guiding and shaping interviews and focus groups heavily reflect the evaluators' understanding and…

Positive Behavioral Interventions and Supports (PBIS) is an evidence-based approach to addressing behavior issues in schools. A significant amount of research has been done to identify the critical features of PBIS. More generally, implementation science points to a specific sequence to ensure that PBIS is implemented with fidelity. This fall…

Dual enrollment is a complex policy issue and is increasingly of interest to state policymakers. Adequate data and data systems are critical to informing dual enrollment policy, yet the data at the federal level and many state data systems suffer from common challenges and lack the capacity to address dual enrollment policy. We review the…

Positive Behavioral Interventions and Supports (PBIS) is an evidence-based approach to addressing behavior issues in schools. A significant amount of research has been done to identify the critical features of PBIS. More generally, implementation science points to a specific sequence to ensure that PBIS is implemented with fidelity. This spring…

The Education Endowment Foundation (EEF)'s mission is to break the link between family income and educational achievement. This is achieved through summarising the best available evidence in plain language, generating new evidence of 'what works' to improve teaching and learning, and supporting teachers and school leaders to use research evidence…

Due to repeated observations of an outcome behavior in N-of-1 or single-case design (SCD) intervention studies, the occurrence of missing scores is inevitable in such studies. Approximately 21% of SCD articles published in five reputable journals between 2015 and 2019 exhibited evidence of missing scores. Missing rates varied by designs, with the…

Analysis of symmetrically predicted endogenous subgroups (ASPES) is an approach to assessing heterogeneity in an ITT effect from a randomized experiment when an intermediate variable (one that is measured after random assignment and before outcomes) is hypothesized to be related to the ITT effect, but is only measured in one group. For example,…

Though we have recently witnessed the "exponential production of digital data to measure, analyze, and predict educational performance" (Salajan & Jules, this volume), there has not been sufficient attention given to the quantitative methods that are used to process and transform this data in order to arrive at findings related to…

While there have been significant developments in the field of inclusive research with people with learning disabilities, there is limited knowledge about the ways inclusive research teams have conducted participatory data analysis. The authors argue this is inherently tied to the sensitivities involved in the inclusive research process. In this…

Policymakers have expressed increased interest in program-level higher education accountability measures as a supplement to, or in place of, institution-level metrics. But it is unclear what these measures should look like. In this report, we assess the ways program-level data could be developed to facilitate federal accountability. Evidence shows…

Supplemental Instruction (SI) is a voluntary, non-remedial, peer-facilitated, course-specific intervention that has been widely demonstrated to increase student success, yet concerns persist regarding the biasing effects of disproportionate participation by already higher-performing students. With a focus on maintaining access for all students, a…

This Working Paper explores the extent to which learning assessment data influence education policymaking according to the available literature. It presents the key conclusions arising from this literature as well as problematic issues which are still open for discussion. It is based on a broader literature review conducted by the UNESCO…

Departments and programmes in higher education are required to participate in an increasing number of programme, course and student assessments. These assessment requirements are also opportunities to develop student skills related to scientific literacy and research, if students are included in the process of developing, administering and…

This response aims to critically evaluate the methodology and aims of the meta-analytic review written by Maggin and Johnson (2014). The present authors systematically provide responses for each of the original criticisms and highlight concerns regarding Maggin and Johnson's methodology, while objectively describing the current state of evidence…