ERIC Number: EJ1470142
Record Type: Journal
Publication Date: 2025
Pages: 11
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-2289-2990
Available Date: 0000-00-00
Employing a Hierarchical Rater Models for Automated Scoring: Scope Review on the Application in Educational Assessment
Akif Avcu
Malaysian Online Journal of Educational Technology, v13 n2 p37-47 2025
This scope-review presents the milestones of how Hierarchical Rater Models (HRMs) become operable to used in automated essay scoring (AES) to improve instructional evaluation. Although essay evaluations--a useful instrument for evaluating higher-order cognitive abilities--have always depended on human raters, concerns regarding rater bias, inconsistency, and scalability have motivated the development of automated systems. By modeling rater behaviors, task complexity, and interaction effects, HRMs handle these issues and offer a strong base to minimize biases and increase dependability. Advances in machine learning and natural language processing (NLP) have helped HRMs to be included into AES systems. Leveraging HRMs to rectify rater biases and guarantee fairness, these systems include language analysis, semantic evaluation, and contextual understanding. This review also includes how Signal Detection Theory (SDT) is being included into HRMs to improve their capacity to assess rater sensitivity and bias and provide understandable results. By including HRMs into AES, feedback quality is improved as well as score accuracy, hence facilitating more exact formative evaluations become possible. Still present, though, are difficulties like computing complexity, dataset availability, and algorithmic bias. The paper emphasizes the possibilities of HRMs in creating fair, high-quality, scalable evaluation systems and supports ongoing research to improve these approaches for various educational environments.
Descriptors: Automation, Scoring, Models, Educational Assessment, Evaluators, Essay Tests, Computer Uses in Education, Statistics
University of Malaya Faculty of Education. Kuala Lumpur 50603 Malaysia. e-mail: info@mojet.net; Web site: http://www.mojet.net
Publication Type: Journal Articles; Information Analyses
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A