A substantial number of randomized trials of educational interventions that have been conducted over the past two decades have produced null results, with either no impact or an unreliable estimate of impact on student achievement or other outcomes of interest. The investment of time and money spent implementing such trials warrants more useful…

This article addresses the gap between what works in research and what works in practice. Currently, research in evidence-based education policy and practice focuses on randomized controlled trials. These can support causal ascriptions ("It worked") but provide little basis for local effectiveness predictions ("It will work…

In a climate of fiscal constraint and cuts to public spending, it is more important than ever to demonstrate the effectiveness of learning programmes. The author carried out an action research project within Norfolk County Council's adult community learning service, where she works as a tutor, with the aim of increasing the impact of reminiscence…

The focus of this chapter is on the evidence to date that documents the impact of PDS engagement on four specific groups of participants: veteran P-12 teachers, university faculty, P-12 school principals, and parents and community members. In reviewing the literature concerning the impact of PDS engagement on these four groups, the authors'…

In the May 2001 "Educational Leadership," Stanley Pogrow wrote another article attacking the Success for All program and comprehensive reform. SFA has been highly evaluated in numerous studies and by 52 individual researchers. Pogrow is a competitor who believes his own program (Higher Order Thinking Skills) has been slighted. (Contains…

This article discusses the use of propensity scoring in experimental program evaluations to estimate impacts for subgroups defined by program features and participants' program experiences. The authors discuss estimation issues and provide specification tests. They also discuss the use of an overlooked data collection design--obtaining predictions…

Success for All, evaluated highly in several cities, has received unfair criticism. Opposing the process for developing, evaluating, and disseminating effective programs to high-poverty schools (particularly highlighting flaws in a tiny subset of studies) merely sows doubt and misinformation about a most promising reform development. (Contains 44…

Documenting treatment integrity is an important issue in research and practice in any discipline concerned with prevention and intervention. However, consensus concerning the dimensions of treatment integrity and how they should be measured has yet to emerge. Advances from three areas in which significant treatment integrity work has taken…

This concluding chapter reviews some of the methodological issues that analysts need to consider when evaluating the success of character-development programs.

Slavin's research that "validated" his Success for All program's effectiveness drives the current push for schoolwide models and comprehensive reform. Other research shows achievement gains for K-1 students only. Concentrating knowledge generation in just a few groups stifles innovative thinking and limits development of alternative…

This paper details the challenges encountered by authors summarizing evidence from a primary study to describe a treatment's effectiveness using an effect size (ES) estimate. Dilemmas that are encountered, including how to calculate and interpret the pertinent standardized mean difference ES for results from studies of various research designs,…

Imported from New Zealand in 1984, Reading Recovery currently operates in 9,800 schools nationwide. Now the much- touted program for teaching at-risk readers is facing questions about its effectiveness and cost. According to one researcher, Reading Recovery temporarily increases the rate of reading development but has little effect on developing…

Group-randomized trials (GRTs)--studies that evaluate the efficacy or effectiveness of interventions occurring at the group level--are increasingly used to assess the effects of school-based prevention programs on behavioral outcomes of children and adolescents. These designs pose many implementation and design challenges for school…

Outlines the pros and cons of Project JOLT, a "Scared Straight" type of program. Raises some theoretical, methodological, and political issues surrounding the Scared Straight approach. Illustrates some of the pitfalls in program evaluation in corrections and makes some recommendations. (Author/RC)

It is often claimed that techniques such as "multimedia" and the use of blended learning environments can be used to achieve powerful interactive pedagogies. Indeed, the advent of easy-to-use multimedia technologies has meant that a plethora of digital learning products is now becoming available. Despite the relative ease-of-use of these new…