This analysis employs a Bayesian framework to estimate the impact of a Cognitive-Behavioral Therapy (CBT) intervention on the recidivism of high-risk people under community supervision. The study relies on the reanalysis of experimental datal using a Bayesian logistic regression model. In doing so, new estimates of programmatic impact were…

In this response, we first show that Simpson's proposed analysis answers a different and less interesting question than ours. We then justify the choice of prior for our Bayes factors calculations, but we also demonstrate that the substantive conclusions of our article are not substantially affected by varying this choice.

A recent paper uses Bayes factors to argue a large minority of rigorous, large-scale education RCTs are "uninformative." The definition of "uninformative" depends on the authors' hypothesis choices for calculating Bayes factors. These arguably overadjust for effect size inflation and involve a fixed prior distribution,…

In the coming years, public programs will capture even more and richer data than they do now, including data from web-based tools used by participants in employment services, from tablet-based educational curricula, and from electronic health records for Medicaid beneficiaries. Program evaluators seeking to take full advantage of these data…

The present article considers a fundamental question in evaluation research: "By how much do program effects vary across sites?" The article first presents a theoretical model of cross-site impact variation and a related estimation model with a random treatment coefficient and fixed site-specific intercepts. This approach eliminates…

Wills and Pothos (2012) reviewed approaches to evaluating formal models of categorization, raising a series of worthwhile issues, challenges, and goals. Unfortunately, in discussing these issues and proposing solutions, Wills and Pothos (2012) did not consider Bayesian methods in any detail. This means not only that their review excludes a major…

Evaluating and monitoring large-scale distance learning programs require different techniques, systems, and analysis methods. This work presents challenges in evaluating and monitoring digital inclusion training programs, considering the aspects inherent in large-scale distance training, and reports an approach based on network and distance…

Propensity score applications are often used to evaluate educational program impact. However, various options are available to estimate both propensity scores and construct comparison groups. This study used a student achievement dataset with commonly available covariates to compare different propensity scoring estimation methods (logistic…

There are two mainstream curricula for international school students at the junior high level: the International Baccalaureate (IB) Middle Years Programme (MYP) and the Cambridge International General Certificate of Secondary Education (IGCSE). The former was developed in the mid-1990s and is currently being relaunched in a 21st-century approach.…

The Multiattribute Utility Model combines subjective goal definition with objective data analysis. Goals are defined, ranked, and weighted. Subjective opinions about their attainment are assigned against decision alternatives considered by the school. Bayesian analysis of data enables revision of prior opinions about the realization of goals…

This study extended Schilling's (2001) study by investigating program commitment among a larger (N = 12), older (M = 16.7 years), and more experienced (M = 5.96 years) sample of participants in the Project Effort Youth Leader Corps. Individual interviews elicited a greater number and specificity of themes. Program-related barriers included…

Trade-offs are implicit in choosing a subjective or objective method for evaluating social programs. The differences between Bayesian and traditional statistics, decision and cost-benefit analysis, and anthropological and traditional case systems illustrate trade-offs in choosing methods because of limited resources. (SLD)

The Bayesian inferential process is modified for use in an aggregate meta-analytic evaluation. Compared with the average effect size meta-analytic approach, the Bayesian approach was more sensitive, more consistent and more powerful. This approach is recommended when primary data are not available and when all evaluations involve comparisons of…

Evaluation is discussed as an information-gathering process. Currently popular evaluation programs are reviewed in relation to decision making and various approaches that seem to contribute to the decision utility of evaluation (e.g. classical approaches, Bayesian approaches, adaptive designs, and model-based evaluations) are described. (Author/AL)