This paper reviews the use of multiple trials, defined as multiple sites or multiple arms in a single evaluation and replications, in evaluating social programs. After defining key terms, the paper discusses the rationales for conducting multiple trials, which include increasing sample size to increase statistical power; identifying the most…

Near-peer teaching, in which students one or more years removed from a student within the same academic program, is utilized broadly in medical school curricula and increasingly in undergraduate anatomy and physiology courses. While the literature on near-peer teaching suggests that the practice is overwhelmingly positive, it is primarily from…

The purpose of this paper is to review and critique a groundbreaking research report. The research article under review in this paper is: "A Big Apple for Educators: New York City's Experiment with Schoolwide Performance Bonuses: Final Evaluation Report" (see ED521920). In 2007-08, a program called the Schoolwide Performance Bonus…

For over a decade, the National Science Foundation's Innovative Technology Experiences for Students and Teachers (ITEST) program has funded researchers and educators to build an understanding of best practices, contexts, and processes contributing to K-12 students' motivation and participation in Science, Technology, Engineering, and Mathematics…

This chapter provides a selective review of some contemporary approaches to program evaluation. One motivation for our review is the recent emergence and increasing use of a particular kind of "program" in applied microeconomic research, the so-called Regression Discontinuity (RD) Design of Thistlethwaite and Campbell (1960). We organize our…

In the current climate of shortages of high-quality engineering graduates, exacerbated by reduced high school enrollments in physics and mathematics, engineering faculties are becoming increasingly aware of the importance of K-12 outreach programs. Such programs can result in students being better prepared for and better informed about engineering…

This article examines theoretical and empirical issues related to the statistical power of impact estimates for experimental evaluations of education programs. The author considers designs where random assignment is conducted at the school, classroom, or student level, and employs a unified analytic framework using statistical methods from the…

Understanding the scientific method fosters the development of critical thinking and logical analysis of information. Additionally, proposing and testing a hypothesis is applicable not only to science, but also to ordinary facts of daily life. Knowing the way science is done and how its results are published is useful for all citizens and…

Offers a critical commentary on theory-based evaluation, stressing its utility as a method of program planning and as an adjunct to experiments but rejecting it as an alternative to experiments. Cites seven reasons for doubting that theory-based evaluations can provide the valid conclusions about a program's causal effects that proponents have…

Although the vast majority of articles cited by Jeanne Chall and Marie Carbo fit the patched-up program evaluation category, the outcomes of nine randomized field experiments are the best evidence available in the research literature. Systematic phonics is an example of the weak instructional treatments presently plaguing education. Includes 16…

It is argued that reports of evaluation activities often leave much to be desired because they fail to portray the real trial-and-error processes of the research study and do not report on the logic in use at the time of the inquiry. (SLD)

Ethnic minority students traditionally pursue degrees and careers in the food and agricultural sciences at rates lower than their non-minority counterparts. To help improve upon this situation, the Food and Agricultural Sciences Institute (FASI) was created to expose academically talented high school students to opportunities within the food and…

This paper examines the major methodological advantages of random assignment for the purpose of estimating the effectiveness of current manpower policy. It also reviews the claimed methodological and ethical objections to experiments. The authors argue that the offsetting gain from experimentation is the inherent reliability of experimental…

Six potential problems with the use of randomized experiments to evaluate programs in the field are addressed. Problems include treatment dilution, treatment contamination or confounding, inaccurate case flow and power estimates, violations of the random assignment processes, changes in the environmental context, and changes in the treatment…

Propositions about what works in program evaluation are presented: (1) evaluators should be involved from the beginning; (2) evaluation should be based on explicit programmatic theory; (3) quasi-experimental designs must have high internal validity; and (4) evaluators must combat the deterioration of experimental control that occurs in…