Online calibration estimates new item parameters alongside previously calibrated items, supporting efficient item replenishment. However, most existing online calibration procedures for Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT) lack mechanisms to ensure content balance during live testing. This limitation can lead to uneven…

Background: The validity of unmonitored online exams has raised concerns about academic integrity and grade inflation, especially given the rise of artificial intelligence-powered tools. Objective: This study evaluates the validity of unmonitored online exams by comparing student performance between two sections of an undergraduate personality…

In this three-study investigation, we applied various approaches to score drawings created in response to both Form A and Form B of the Torrance Tests of Creative Thinking-Figural (broadly TTCT-F) as well as the Multi-Trial Creative Ideation task (MTCI). We focused on TTCT-F in Study 1, and utilizing a random forest classifier, we achieved 79% and…

Computer adaptive testing (CAT) is an increasingly common mode of test administration offering improved test security, better measurement precision, and the potential for shorter testing experiences. This article presents a new item selection algorithm based on a generalized objective function to support multiple types of testing conditions and…

Recent developments in the use of large-language models have led to substantial improvements in the accuracy of content-based automated scoring of free-text responses. The reported accuracy levels suggest that automated systems could have widespread applicability in assessment. However, before they are used in operational testing, other aspects of…

Unproctored Computerized Adaptive Testing (CAT) is gaining traction due to its convenience, flexibility, and scalability, particularly in high-stakes assessments. However, the lack of proctor can give rise to aberrant testing behavior. These behaviors can impair the validity of test scores. This paper explores the use of a verification test to…

Background/purpose: This study addresses the problem of demotivation generated by traditional assessment methods, which are often standardized, unengaging, and ill-suited to individual differences. In an increasingly digitized educational context, the primary objective is to assess the ability of an adaptive assessment system, developed on the…

Example-generation tasks have been suggested as an effective way to both promote students' learning of mathematics and assess students' understanding of concepts. E-assessment offers the potential to use example-generation tasks with large groups of students, but there has been little research on this approach so far. Across two studies, we…

This research set out to enhance our understanding of the exam techniques and types of written annotations or markings that learners may wish to use to support their thinking when taking digital multiple-choice exams. Additionally, we aimed to further explore issues around the factors that contribute to learners writing less rough work and…

Purpose: The PEDI-CAT (ASD) is used to assess functioning of children and youth on the autism spectrum; however, current psychometric evidence is limited. This study aimed to explore the reliability, validity and acceptability of the PEDI-CAT (ASD) using a large Australian sample. Methods: Caregivers of 134 children and youth on the spectrum…

In computerized adaptive testing, item exposure control methods are often used to provide a more balanced usage of the item pool. Many of the most popular methods, including the restricted method (Revuelta and Ponsoda), use a single maximum exposure rate to limit the proportion of times that each item is administered. However, Barrada et al.…

This study leverages data from direct assessments of learning (AoL) to build a dynamic model of student performance in competency exams related to computer technology. The analysis reveals three key predictors that strongly influence student success: performance on a practice exam, whether or not a student engaged in practice testing beforehand,…

The Ohio Department of Education and Workforce, in response to Senate Bill 168 (135th General Assembly), initiated a pilot program in the 2024-2025 school year to test the feasibility of remotely administered and proctored state assessments. This pilot aimed to explore the potential of remote testing to enhance flexibility and accessibility for…

Prompts play a crucial role in eliciting accurate outputs from large language models (LLMs). This study examines the effectiveness of an automatic prompt engineering (APE) framework for automatic scoring in educational measurement. We collected constructed-response data from 930 students across 11 items and used human scores as the true labels. A…

The significance of teacher's assessment literacy (AL) was originally captured by the 1990 standards for teacher's competence in educational assessment. Competence in assessment has changed with the widespread use of recent technology advancements in educational assessment. Consequently, new measures are needed to measure Teacher Assessment…