NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1279691
Record Type: Journal
Publication Date: 2020
Pages: 13
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1939-1382
EISSN: N/A
Available Date: N/A
The Effects of Dispersion and Reciprocity on Assessment Fidelity in Peer-Review Systems: A Simulation Study
IEEE Transactions on Learning Technologies, v13 n3 p580-592 Jul-Sep 2020
Over the last 20 years, online peer review and assessment have become widely used and well-researched practices in education. Their use increased, especially with the proliferation of nonconventional large-scale and online modes of teaching and learning, such as Massive Open Online Courses (MOOCs). A well-designed peer-review system is expected to produce valid and reliable assessments of the artifacts created by participants. These systems notably vary in designs, particularly in the structure of the peer-review networks, i.e., how participants are linked to each other as creators and reviewers. To date, little research has been done on how different network structures impact a system's ability to accurately assess the quality of the evaluated artifacts. We begin to address this gap, using a simulation approach to demonstrate that two network characteristics--dispersion and reciprocity--may indeed affect assessment fidelity in peer-review systems in conjunction with other design choices, such as evaluation scale and aggregation method. We also outline directions for further investigations of peer-review systems design.
Institute of Electrical and Electronics Engineers, Inc. 445 Hoes Lane, Piscataway, NJ 08854. Tel: 732-981-0060; Web site: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=4620076
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A