When describing motion in physics, the selection of a frame of reference is crucial. The graph of a moving object can look quite different based on the frame of reference. In recent years, various tests have been developed to assess the interpretation of kinematic graphs, but none of these tests have specifically addressed differences in reference…

Multiple-choice questions (MCQs) are widely used in digital learning systems, as they allow for automating the assessment process. However, owing to the increased digital literacy of students and the advent of social media platforms, MCQ tests are widely shared online, and teachers are continuously challenged to create new questions, which is an…

Multi-angle question answering models have recently been proposed that promise to perform related tasks like question generation. However, performance on related tasks has not been thoroughly studied. We investigate a leading model called Macaw on the task of multiple choice question generation and evaluate its performance on three angles that…

The intricate relationship between representational competence and content knowledge in organic chemistry has been widely debated, and the ways in which representations contribute to task difficulty, particularly in assessment, remain unclear. This paper presents a multiple-choice test instrument for assessing individuals' knowledge of fundamental…

In the Attentional Boost Effect (ABE), words or images encoded with to-be-detected target squares are later recognized better than words or images encoded with to-be-ignored distractor squares. The present study sought to determine whether the ABE enhanced the encoding of the item-specific and relational properties of the studied words by using…

Large introductory classes, with their expansive curriculum, demand assessment strategies that blend efficiency with reliability, prompting the consideration of multiple-choice (MC) tests as a viable option. Crafting a high-quality MC test, however, necessitates a meticulous process involving reflection on assessment format appropriateness, test…

Many undergraduate chemistry students hold alternate conceptions related to resonance--an important and fundamental topic of organic chemistry. To help address these alternate conceptions, an organic chemistry instructor could administer the resonance concept inventory (RCI), which is a multiple-choice assessment that was designed to identify…

Sound waves are one of the important topics studied in physics. However, students' graph representation is still low, leading to their low concept understanding of physics learning. Therefore, it is necessary to optimize the learning process to improve students' graph representation by providing a valid and reliable graph representation test…

This study aims to delve into the process and perceptions of pre-service teachers as they engage in generating multiple-choice questions with the assistance of generative AI tools. Adopting a single-case study design, the research involved the participation of 35 pre-service teachers. The participants were tasked with utilizing generative AI tools…

This article is concerned with the statistical detection of copying on multiple-choice exams. As an alternative to existing permutation- and model-based copy-detection approaches, a simple randomization p-value (RP) test is proposed. The RP test, which is based on an intuitive match-score statistic, makes no assumptions about the distribution of…

The testlet design is very popular in educational and psychological assessments. This article proposes a new cognitive diagnosis model, the multiple-choice cognitive diagnostic testlet (MC-CDT) model for tests using testlets consisting of MC items. The MC-CDT model uses the original examinees' responses to MC items instead of dichotomously scored…

In tertiary mathematics courses, students often have difficulties acquiring an understanding of the mathematical concepts covered. One approach to address this problem is to implement so-called Concept-Tests. These are multiple-choice questions whose distractors represent common problems and misconceptions related to the concepts. While there…

Digital administration of assessments allows for the collection of process data indices, such as response time, which can serve as indicators of rapid-guessing and examinee test-taking effort. Setting a time threshold is essential to distinguish effortful from effortless behavior using item response times. Threshold identification methods may…

A better understanding of how distractor features influence the plausibility of distractors is essential for an efficient multiple-choice (MC) item construction in educational assessment. The plausibility of distractors has a major influence on the psychometric characteristics of MC items. Our analysis utilizes the nominal categories model to…

Typical approaches to test and item development are rooted in the "Standards for Educational and Psychological Testing." Culturally responsive and antiracist assessment practices are two new processes that challenge the typical process noted in the "Standards," incorporating critical race theory and cultural responsiveness into…