The recognition that researcher discretion coupled with unconscious biases and motivated reasoning sometimes leads to false findings ("p-hacking") led to the broad embrace of study preregistration and other open-science practices in experimental research. Paradoxically, the preregistration of quasi-experimental studies remains uncommon…

In recent years, countries including the UK and USA have seen advancements in the use of Randomised Controlled Trials in education, progress that has not been mirrored in Ireland. Ireland does not have a strong tradition of using experimental or quasi-experimental evaluation designs for monitoring and evaluation of education policy despite…

Problem definition in program planning and evaluation is rarely problematized. In this article, I discuss why the lack of problem problematization is itself problematic--in other words, why treating problems as self-evident can pose a risk for evaluation practice. Then, to help avoid such risks, I suggest Carol Bacchi's "What's the Problem…

Crisis intervention programs have been developed to help educators be better prepared to prevent and respond to students in crisis. Schools spend millions of dollars for the use of these commercial programs, many of which are based on the principles of the Cycle of Acting Out Behavior (Colvin & Scott, 2015), which is considered the "gold…

Premised on the idea that evaluators should be familiar with a range of approaches to program modifications, I review several existing approaches and then describe another, less well-recognized option. In this newer option, evaluators work with others to identify potentially needed adaptations for select program aspects "in advance." In…

Children face increased demands for interpersonal as well as learning-related social skills, especially by the vulnerable stage of school entry, due to the more structured setting, new academic requirements, and the fact that children are supposed to interact successfully within a larger and heterogeneous peer group. Although a plethora of…

Over the last few decades, there has been an increase in the number of large federally funded transdisciplinary programs and initiatives. Scholars have identified a need to develop frameworks, methodologies, and tools to evaluate the effectiveness of these large collaborative initiatives, providing precise ways to understand and assess the…

Realist evaluation (RE) is a research design increasingly used in program evaluation, that aims to explore and understand the influence of context and underlying mechanisms on intervention or program outcomes. Several methodological challenges, however, are associated with this approach. This article summarizes RE key principles and examines some…

School consultants often use problem-solving consultation when working with educators and caregivers to support the behavioral and academic outcomes of students. Considering some of the well-known limitations to in-person service delivery, consultants have increasingly relied on telecommunications as a medium through which to deliver…

This essay highlights the value of an applied methodology course in program evaluation in the education doctorate program by exploring several benefits that it offers to enhance a doctoral student's ability to solve complex problems of practice. Observations and recommendations are made based on designing and teaching two cohorts of EdD students…

In this response, we first show that Simpson's proposed analysis answers a different and less interesting question than ours. We then justify the choice of prior for our Bayes factors calculations, but we also demonstrate that the substantive conclusions of our article are not substantially affected by varying this choice.

This paper describes the context, processes and issues experienced over 5 years in which a RCT was carried out to evaluate a programme for children aged 7-8 who were struggling with their reading. Its specific aim is to illuminate questions about the design of complex teaching approaches and their evaluation using an RCT. This covers the early…

"The Mentoring Toolkit 2.0: Resources for Developing Programs for Incarcerated Youth" provides information, program descriptions, and links to important resources that can assist juvenile correctional facilities and other organizations to design effective mentoring programs for neglected and delinquent youth, particularly those who are…

This is the executive summary for the report "North Carolina Pre-K Pyramid Model Evaluation Study: Implementation and Outcomes Evaluation Study Final Results." The report is an assessment of a pilot implementation of the NC Preschool Pyramid Model (Pyramid Model), a social-emotional coaching program to support NC Pre-K teachers to engage…

Despite the requirement that teachers implement evidence-based instruction in their classrooms, a significant research-to-practice gap persists. Far too often, teachers resort to quick fixes found through online searches or rely on conventional wisdom to make instructional decisions. This is no surprise as identifying evidence-based interventions…