In this article, we discuss the methodological implications of data and theory integration for Theory-Based Evaluation (TBE). TBE is a family of approaches to program evaluation that use program theories as instruments to answer questions about whether, how, and why a program works. Some of the groundwork about TBE has expressed the idea that a…

Monitoring, evaluation, and research activities generate important data, but they often fail to change policies or programs. In addition, local program staff and partners often feel disconnected from these activities, which undermines their ownership of data and results. To bridge the gaps between monitoring, evaluation, and research and to give…

Qualitative comparative analysis (QCA) is gaining ground in evaluation circles, but the number of applications is still limited. In this article, we consider the challenges that can emerge during a QCA evaluation by drawing on our experience of conducting one in the field of development cooperation. For each stage of the evaluation process, we…

Bhutan's overarching development paradigm of Gross National Happiness (GNH) promotes a harmonious balance between material and non-material dimensions. But Bhutan's evaluation practice has not yet adopted the principles of GNH, preventing evaluation findings and recommendations from aligning with the priorities of GNH. This article makes the case…

We assessed whether an adaptation of value-added analysis (VAA) can provide evidence on the relative effectiveness of interventions implemented in a large school district. We analyzed two datasets, one documenting interventions received by underperforming students, and one documenting interventions received by students in schools benefiting from…

The purpose of this paper is to introduce the Five-Level Qualitative Data Analysis (5LQDA) method for ATLAS.ti as a way to intentionally design methodological approaches applicable to the field of evaluation. To demonstrate my analytical process using ATLAS.ti, I use examples from an existing evaluation of a STEM Peer Learning Assistant program.…

In the coming years, public programs will capture even more and richer data than they do now, including data from web-based tools used by participants in employment services, from tablet-based educational curricula, and from electronic health records for Medicaid beneficiaries. Program evaluators seeking to take full advantage of these data…

This article examines factors that could have influenced whether evaluations of U.S. government--funded foreign assistance programs completed in 2015 had considered unintended consequences. Logit regression models indicate that the odds of considering unintended consequences were increased when all or most of seven standard data collection methods…

Interest in the regression discontinuity (RD) design as an alternative to randomized control trials (RCTs) has grown in recent years. There is little practical guidance, however, on conditions that would lead to a successful RD evaluation or the utility of studies with underpowered RD designs. This article describes the use of RD design to…

Project schedules are logic models that focus on the timing of program activities. Value derives from the fact that schedule changes are not random. Why they occur, and how long they last, can reveal information that would not be easily revealed with other approaches to evaluation. Also, using project schedules as logic models forges a strong link…

The growth in the availability of longitudinal data--data collected over time on the same individuals--as part of program evaluations has opened up exciting possibilities for evaluators to ask more nuanced questions about how individuals' outcomes change over time. However, in order to leverage longitudinal data to glean these important insights,…

Chronic diseases with behavioral risk factors are now the leading causes of death in the United States. A national Behavioral Risk Factor Surveillance System (BRFSS) monitors those risk factors; however, there is a need for national and state evaluations of chronic disease surveillance systems. The Department of Health and Human Services/Centers…

The "performance movement" has been a subject of enthusiasm and frustration for evaluators. Performance measurement, data analytics, and program evaluation have been treated as different tasks, and those addressing them speak their own languages in their own circles. We suggest that situating performance measurement and data analytics…

This article presents discussion and recommendations on approaches to retrospectively evaluating development interventions in the long term through a systems lens. It is based on experiences from the implementation of an 18-month study to investigate the impact of development interventions on economic and social change over a 40-year period in the…

Several analytic strategies exist for opening up the "black box" to reveal more about what drives policy and program impacts. This article focuses on one of these strategies: the Analysis of Symmetrically-Predicted Endogenous Subgroups (ASPES). ASPES uses exogenous baseline data to identify endogenously-defined subgroups, keeping the…