ERIC Number: EJ1360556
Record Type: Journal
Publication Date: 2022
Pages: 15
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-2056-4880
EISSN: N/A
Available Date: N/A
Time-Dependent Metrics to Assess Performance Prediction Systems
International Journal of Information and Learning Technology, v39 n5 p451-465 2022
Purpose: The goal is to assess performance prediction systems (PPS) that are used to assist at-risk learners. Design/methodology/approach: The authors propose time-dependent metrics including earliness and stability. The authors investigate the relationships between the various temporal metrics and the precision metrics in order to identify the key earliness points in the prediction process. Authors propose an algorithm for computing earliness. Furthermore, the authors propose using an earliness-stability score (ESS) to investigate the relationship between the earliness of a classifier and its stability. The ESS is used to examine the trade-off between only time-dependent metrics. The aim is to compare its use to the earliness-accuracy score (EAS). Findings: Stability and accuracy are proportional when the system's accuracy increases or decreases over time. However, when the accuracy stagnates or varies slightly, the system's stability is decreasing rather than stagnating. As a result, the use of ESS and EAS is complementary and allows for a better definition of the point of earliness in time by studying the relation-ship between earliness and accuracy on the one hand and earliness and stability on the other. Originality/value: When evaluating the performance of PPS, the temporal dimension is an important factor that is overlooked by traditional measures current metrics are not well suited to assessing PPS's ability to predict correctly at the earliest, as well as monitoring predictions stability and evolution over time. Thus, in this work, the authors propose time-dependent metrics, including earliness, stability and the trade-offs, with objective to assess PPS over time.
Descriptors: Performance, Prediction, Student Evaluation, At Risk Students, Time, Identification, Accuracy, Learning Analytics, Artificial Intelligence, Elementary Secondary Education, Algorithms, Scores
Emerald Publishing Limited. Howard House, Wagon Lane, Bingley, West Yorkshire, BD16 1WA, UK. Tel: +44-1274-777700; Fax: +44-1274-785201; e-mail: emerald@emeraldinsight.com; Web site: http://www.emerald.com/insight
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A