Assessment is viewed as an important means to understand learners' performance in the learning process. A good assessment method is based on high-quality examination questions. However, generating high-quality examination questions manually by teachers is a time-consuming task, and it is not easy for students to obtain question banks. To solve…

Designing and constructing pedagogical tests that contain items (i.e. questions) which measure various types of skills for different levels of students equitably is a challenging task. Teachers and item writers alike need to ensure that the quality of assessment materials is consistent, if student evaluations are to be objective and effective.…

This case study reports on a co-creation initiative that explored the use of generative artificial intelligence (AI) technologies in the context of higher education assessments. The AI Co-Creators project aimed to promote dialogue between students and staff on the impact of AI with the objective of using AI tools more effectively. The case study…

This research aimed to develop and validate a rubric to assess Artificial Intelligence (AI) chatbots' effectiveness in accomplishing tasks, particularly within educational contexts. Given the rapidly growing integration of AI in various sectors, including education, a systematic and robust tool for evaluating AI chatbot performance is essential.…

Large language models (LLMs), such as ChatGPT, are becoming increasingly prominent. Their use is becoming more and more popular to assist with simple tasks, such as summarizing documents, translating languages, rephrasing sentences, or answering questions. Reports like McKinsey's (Chui, & Yee, 2023) estimate that by implementing LLMs,…

The rapid advancement of artificial intelligence (AI) technologies, particularly OpenAI's ChatGPT, has significantly impacted higher education institutions (HEIs), offering opportunities and challenges. While these tools enhance personalized learning and content generation, they threaten academic integrity, especially in assessment environments.…

Multiple choice questions are traditionally expensive to produce. Recent advances in large language models (LLMs) have led to fine-tuned LLMs that generate questions competitive with human-authored questions. However, the relative capabilities of ChatGPT-family models have not yet been established for this task. We present a carefully-controlled…

In the last two years, there has been a significant increase in research studies on ChatGPT and its role in educational assessment. However, there is no comprehensive systematic literature review (SLR) on the potential use of ChatGPT for educational assessment, particularly with a focus on its practices and limitations. To address this gap, our…

The interpretations of test scores in secure, high-stakes environments are dependent on several assumptions, one of which is that examinee responses to items are independent and no enemy items are included on the same forms. This paper documents the development and implementation of a C#-based application that uses Natural Language Processing…

Automatic item generation can rapidly generate large volumes of exam items, but this creates challenges for assembly of exams which aim to include syntactically diverse items. First, we demonstrate a diminishing marginal syntactic return for automatic item generation using a saturation detection approach. This analysis can help users of automatic…

Recent AI-based language learning research highlights learners' crucial role, yet university learner readiness in ChatGPT-based English learning remains unexplored. Accordingly, this current research attempted to develop and validate a tool to evaluate university learner readiness for ChatGPT-assisted English learning (LRCEL) to address the…

Following the burgeoning growth of artificial intelligence (AI) and machine learning (ML) applications in language assessment in recent years, the meteoric rise of ChatGPT and its sweeping applications in almost every sector have left us in awe, scrambling to catch up by developing theories and best practices. This special issue features studies…

Multiple-choice quizzes (MCQs) are a popular form of assessment. A rapid shift to online assessment during the COVID-19 pandemic in 2020, drove the uptake of MCQs, yet limited invigilation and wide access to material on the internet allow students to solve the questions via internet search. ChatGPT, an artificial intelligence (AI) agent trained on…

In the swiftly evolving landscape of education, the fusion of Artificial Intelligence's ingenuity with the dynamic capabilities of chat-bot technology has ignited a transformative paradigm shift. This convergence is not merely a technological integration but a profound reshaping of the fundamental principles of pedagogy, fundamentally redefining…

Oral reading fluency probes are essential for reading assessment, intervention, and progress monitoring. Due to the limited options for choosing oral reading fluency probes, it is important to utilize all available resources such as generative artificial intelligence (AI) like ChatGPT to create oral reading fluency probes. The purpose of this…