Peer review is integral to the evaluation of grant proposals. Reviewer perceptions and characteristics have received limited study, especially their associations with reviewers' evaluations. This mixed methods study analyzed the unstructured comments of 270 experienced peer reviewers after they scored proposals based on mock overall evaluations…

Scientific research relies on funding, and everyone who conducts research must become adept at funding their research. This book explains how to attain the number one source of research funding: grants. Readers will learn how to prepare grant proposals, how and when to interact with funding institutions, how to interpret and respond to peer review…

Inter-rater reliability (IRR), which is a prerequisite of high-quality ratings and assessments, may be affected by contextual variables, such as the rater's or ratee's gender, major, or experience. Identification of such heterogeneity sources in IRR is important for the implementation of policies with the potential to decrease measurement error…

The allocation of scientific funding through grant programs is crucial for research advancement. While independent peer panels typically handle evaluations, their decisions can lean on personal preferences that go beyond the stated criteria, leading to inconsistencies and potential biases. Given these concerns, our study employs a novel method,…

Grant peer review is a foundational component of scientific research. In the context of grant review meetings, the review process is a collaborative, socially mediated, locally constructed decision-making task. The current study examines how collaborative discussion affects reviewers' scores of grant proposals, how different review panels score…

While Elizabeth Barrett Browning counted 25 ways in which she loves her husband in her poem, "How Do I Love Thee? Let me Count the Ways," we identified only eight ways to evaluate the potential for success of a federal research grant proposal. This may be surprising, as it seems upon initial glance of the review criteria used by various…

The demand for greater public accountability is changing the nature of ex ante peer review at public science agencies worldwide. Based on a four year research project, this essay examines these changes through an analysis of the process of grant proposal review at two US public science agencies, the National Institutes of Health (NIH) and the…

Career grants are an important instrument for selecting and stimulating the next generation of leading researchers. Earlier research has mainly focused on the relation between past performance and success. In this study we investigate how the selection process takes place. More specifically, we investigate which quality dimensions (of the…

Peer review is valued in higher education, but also widely criticized in terms of potential biases, particularly gender. We evaluate gender differences in peer reviews of grant applications, extending Bornmann, Mutz, and Daniel's meta-analyses that reported small gender differences in favor of men (d = 0.04), but a substantial heterogeneity in…

Peer review is an umbrella term that refers to a class of selection and oversight practices, including the familiar mechanisms of the review of proposals submitted for funding, of manuscripts for scholarly publications, and of personnel qualifications and portfolios for selection and promotion. Peer review has long been a cornerstone of modern…

The procedural changes identified in analyzing the 1979 Marquette University internal grants competition had the positive effect of improving the reliability of the peer review ratings. Improved reliability results in greater confidence in the peer review process on the part of the applicants and the reviewers. (MLW)

Presents a sequence of assignments that provides an inductive method for preparing students to write a formal proposal by imitating the grant review and proposal process. Describes the process and results of peer review of the resulting proposals.(HTH)

Presents a historical view of the development of the system used at the National Science Foundation for judging proposals sent to the agency for financial support for fundamental research. Controversies that challenged established procedures are described; political influences and policy changes are examined; and the peer review process is…

This study evaluated the effect of training on improving the reliability of the peer review process by determining whether or not training made a difference in the variability among reviewers' scores and documentation provided to support the scores. Different levels of training were provided for participants in the peer review process who were…

This report presents the results of a survey conducted by the National Science Foundation (NSF) in late 1986 of more than 14,000 applicants whose proposals for research support had been awarded or declined during fiscal year 1985. The principal purpose was to seek the views of individual investigators at academic institutions about NSF's…