Rapid approaches are essential when resources are limited and when findings are required in real-time to inform decisions. Limitations exist in their design and implementation, which can lead to a reduced level of trust in findings. This review sought to map the methods used across rapid evaluations and research to facilitate timeliness and…

The recognition that researcher discretion coupled with unconscious biases and motivated reasoning sometimes leads to false findings ("p-hacking") led to the broad embrace of study preregistration and other open-science practices in experimental research. Paradoxically, the preregistration of quasi-experimental studies remains uncommon…

Research synthesis of evaluation findings is a multistep process. An investigator identifies a research question, acquires the relevant literature, codes findings from that literature, and analyzes the coded data to estimate the average treatment effect and its distribution in a population of interest. The process of estimating the average…

Evaluations that combine social science and law have tremendous potential to illuminate the effects of governmental policies and yield insights into how effectively policy makers' efforts achieve their aims. This potential is infrequently achieved, however, because such interdisciplinary research contains often overlooked substantive and…

Administered a survey to 606 National Science Foundation-funded principal investigators and their institutions' representatives (n=91) to study responses to unethical research behavior. Scientists appear to perceive that they uphold their responsibility to respond through disclosures in the research community; administrators propose to report such…

This article addresses the issue of falling response rates in telephone surveys. To better understand and maintain respondent goodwill, concepts of psychological contract and respondent expectations are introduced and explored. Results of the qualitative study show that respondent expectations are not only socially contingent but also…

Cross-cultural adaptation of study instruments is a difficult, time-consuming, but arguably cost-effective process. If conducted properly, it has the advantage that the translated study instruments are accurate, easy to understand, accessible, and culturally appropriate to the target audience and produce reliable and valid data. This article…

This article reviews the agricultural research evaluation literature in an effort to assess the extent of coverage in this area. Coverage breakdowns with respect to evaluation approaches, types of research, and products and production processes are presented. Results suggest that there has been limited coverage of research undertaken in…

The purposes of this article are, first, to argue that analyses based on level of treatment implementation can lead to biased estimates of treatment effects, and second, to discuss alternatives to this approach. (PN)

Risks of life on the street caused by inclement weather, harassment, and assault threaten the unsheltered homeless population. We address some challenges of enumerating the street homeless population by testing a novel capture-recapture (CR) estimation approach that models individuals' intermittent daytime visibility. We tested walking and…

The normative implications of applying significance tests in evaluation research are examined. The authors conclude that evaluators often make normative decisions, based on the traditional .05 significance level in studies with small samples. Additional reporting of the magnitude of impact, the significance level, and the power of the test is…

Two studies of the cost-benefit analysis of correctional sentences are reviewed, which present conflicting results. Although such analysis has not determined a way to implement the selective incapacitation of criminals, it has identified the critical determinants of the decision to sentence criminals to prison or probation. (SLD)

Selectivity problems are discussed in terms of a general model that is estimated by the maximum likelihood method. Both single-group and multiple-group analyses are considered. An extension of the general model to latent variable models is discussed. (Author/PN)

Despite a growing literature on evaluation methods suitable for applied settings, few investigators address the separation of service and research. This article recognizes obstacles to field studies. The authors specify practical strategies for entering service environments and for designing, implementing, and measuring an experimental…

Correction for guessing does not fulfill its intended function when test takers who have nothing to gain from scoring will respond randomly when they could have answered correctly had they tried. Raw scores underestimate abilities. If random guessing is more prevalent in the control group, correction for guessing inflates treatment effects.…