ETS scientists developed a human-centered AI (HAI) framework that combines data on how students interact with assessments--such as task navigation and time spent--with their performance, providing deeper insights into student performance in large-scale assessments.

This study explores Hong Kong higher education students' perceptions of E-assessment systems, focusing on factors shaping acceptance of E-examinations over traditional formats. Quantitative analysis of 107 respondents reveals significant positive correlations between diverse pre-exam guidance (e.g., tutorials) and key system features (e.g.,…

Automated assessment is well-established within computer science courses but largely absent from human--computer interaction courses. Automating the assessment of human--computer interaction (HCI) is challenging because the coursework tends not to be computational but rather highly creative, such as designing and implementing interactive…

"A Practical Guide to Language Assessment" helps educators at every level redefine their approach to language assessment. Grounded in extensive research and aligned with the latest advances in language education, this comprehensive guide introduces foundational concepts and explores key principles in test development and item writing.…

Previous research has shown that phonological awareness is one of the most important prerequisites for early reading. Monitoring its development requires reliable, easy-to-use instruments especially in the last years of kindergarten. The present study aims to explore the potential for assessing phonological awareness and some of its subskills…

Background/purpose: This study addresses the problem of demotivation generated by traditional assessment methods, which are often standardized, unengaging, and ill-suited to individual differences. In an increasingly digitized educational context, the primary objective is to assess the ability of an adaptive assessment system, developed on the…

Example-generation tasks have been suggested as an effective way to both promote students' learning of mathematics and assess students' understanding of concepts. E-assessment offers the potential to use example-generation tasks with large groups of students, but there has been little research on this approach so far. Across two studies, we…

This study explored the effectiveness of extended time (ET) accommodations in the 2017 NAEP Grade 8 Mathematics assessment to enhance educational equity. Analyzing NAEP process data through an XGBoost model, we examined if early interactions with assessment items could predict students' likelihood of requiring ET by identifying those who received…

Arithmetic fluency is regarded as a foundational math skill, typically measured as a single construct with pencil-and-paper-based timed assessments. We introduce a tablet-based assessment of single-digit fluency that captures individual trial response times across several embedded experimental contrasts of interest. A large (n = 824) cohort of…

The integration of generative artificial intelligence (GAI) into educational settings offers unprecedented opportunities to enhance the efficiency of teaching and the effectiveness of learning, particularly within online platforms. This study evaluates the development and application of a customized GAI-powered teaching assistant, trained…

Background/purpose: Artificial Intelligence (AI) is increasingly shaping assessment practices in higher education, promising faster feedback and reduced instructor workload while also raising concerns about fairness and transparency. This study examines how AI technologies are transforming assessment processes and the experiences of stakeholders.…

In recent years, the application of explainability techniques to automated essay scoring and automated short-answer grading (ASAG) models, particularly those based on transformer architectures, has gained significant attention. However, the reliability and consistency of these techniques remain underexplored. This study systematically investigates…

Purpose: Academic integrity is vital to the success and sustainability of the academic project and particularly critical in the training of ethical and informed health professionals. Yet studies have found that cheating in online exams was commonplace during the COVID-19 pandemic. With the increased use of online and blended learning…

The convenience and flexibility of online assessments can be beneficial in a variety of ways, but they can also pose risks and challenges, such as potential academic dishonesty by students. This study included 73 master's and doctoral students and investigated the relationship among their attitudes, experiences, and performance in an online…

The study highlights ChatGPT-4's potential in educational settings for the evaluation of university students' open-ended written examination responses. ChatGPT-4 evaluated 54 written responses, ranging from 24 to 256 words in English. It assessed each response using five criteria and assigned a grade on a six-point scale from fail to excellent,…