NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED287888
Record Type: Non-Journal
Publication Date: 1987-Jul
Pages: 22
Abstractor: N/A
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Off-Setting Differences in Reviewer Stringency.
Cason, Carolyn L.; And Others
A total of 1,071 rating sheets were completed by individual reviewers evaluating abstracts submitted to Sigma Theta Tau International Honor Society for the International Research Congress in Edinburgh, Scotland. Only 972 sheets contained usable data. Reviewers indicated a total of 12 ratings with possible comments for each abstract on machine scannable rating sheets specifically prepared for this use. Application of Cason and Cason's simplified model of their deterministic theory to reviews of abstracts resulted in good fit of the model to the data, and detection of significant inter-reviewer differences in individual reviewer's stringency. The availability of calibrated ratings, i.e. those from which these reviewer differences had been removed, greatly eased the task of the Abstract Selection Committee. The Committee no longer needed to deal with ratings which were confounded with variation in reviewer standards. Abstract selection was based on ratings that more accurately reflected the true quality of the abstract without regard to who happened to have reviewed it. When the Abstract Selection Committee used these calibrated ratings in making selection decisions, there were great improvements in both the reliability (from .579 to .810) and validity (from .485 to .742) of the peer review process. (Author/KSA)
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Audience: Researchers
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A