Recent advances in automated scoring technology have made it practical to replace multiple-choice questions (MCQs) with short-answer questions (SAQs) in large-scale, high-stakes assessments. However, most previous research comparing these formats has used small examinee samples testing under low-stakes conditions. Additionally, previous studies…

This study aimed to address the need for a comprehensive assessment tool to evaluate the mathematical abilities of first-grade students through cognitive diagnostic assessment (CDA). The primary challenge involved in this endeavor was to delineate the specific cognitive skills and sub-skills pertinent to first-grade mathematics (FG-M) and to…

This study presents the development and validation of a novel multiple-choice test designed to assess university students' conceptual understanding of electric field, force, and flux. The test of understanding of electric field, force, and flux was constructed based on the results of previous studies using a phenomenographic approach to classify…

This paper describes the design and evaluation of the Organic chemistry Representational Competence Assessment (ORCA). Grounded in Kozma and Russell's representational competence framework, the ORCA measures the learner's ability to "interpret," "translate," and "use" six commonly used representations of molecular…

By design, randomized experiments (XPs) rule out bias from confounded selection of participants into conditions. Quasi-experiments (QEs) are often considered second-best because they do not share this benefit. However, when results from XPs are used to generalize causal impacts, the benefit from unconfounded selection into conditions may be offset…

Research shows that there are multiple benefits to giving college students oral rather than written exams. However, studies that examine, describe, and differentiate how students view their oral exams were never found in a literature search. The purpose of this study was to use Q methodology [Q] to describe the divergent student views about taking…

Exam anxiety has been well-documented to reduce student performance in undergraduate biology courses, especially for students from marginalized groups, which can contribute to achievement gaps. Our exploratory study surveyed 61 undergraduate biology students to better understand how exams affect their anxiety levels, focusing on the impact of exam…

Several researchers have called for schools to interpret universal screening results using posterior probabilities. Following this recommendation could require schools to move away from direct-route, single-measure screening unless base rates of risk fall within a narrow range. In this descriptive study, we investigated two questions surrounding…

In this study, an attempt was made to develop a valid and reliable measurement tool to determine teachers' self-efficacy levels for teaching metacognitive listening strategies. The study group consisted of 205 teachers for EFA and 248 teachers for CFA. As a result of the analyzes, a scale consisting of 16 items with 4 factors was developed. It was…

There are a lot of very interesting scientific concepts to learn in natural and social science. The initial concepts that the student possesses may contradict the actual concepts, which is what causes misconceptions. Misconceptions are identified using misconception detection test tools. In fact, the development of the use of diagnostic test…

Considering the crucial role of education in offering mitigation and adaptation strategies for climate change, there is a clear need for objective tools to assess its impact on the understanding of the issue among secondary school students. This paper describes the methodological design used to build and validate an instrument that explores…

In this paper we will explore an application, made by us, that can generate physics grid tests using artificial intelligence. The application analyzes the response from the large language model in the required format recognized by the application and writes it to a file that will be accepted by the test builder application. The application creates…

Test items with problematic options often require revision to improve their psychometric properties. When an option is identified as ambiguous or nonfunctioning, the traditional approach involves removing the option and conducting another field test to gather new response data--a process that, while effective, is resource-intensive. This study…

In order to ensure content validity by covering a broad range of content domains, the testing times of some educational large-scale assessments last up to a total of two hours or more. Performance decline over the course of taking the test has been extensively documented in the literature. It can occur due to increases in the numbers of: (a)…

In sociolinguistic contexts where standardized languages coexist with regional dialects, the study of differential item functioning is a valuable tool for examining certain linguistic uses or varieties as threats to score validity. From an ecological perspective, this paper describes three stages in the study of differential item functioning…