Test developers and psychometricians have historically examined measurement bias and differential item functioning (DIF) across a single categorical variable (e.g., gender), independently of other variables (e.g., race, age, etc.). This is problematic when more complex forms of measurement bias may adversely affect test responses and, ultimately,…

Traditional estimators of reliability such as coefficients alpha, theta, omega, and rho (maximal reliability) are prone to give radical underestimates of reliability for the tests common when testing educational achievement. These tests are often structured by widely deviating item difficulties. This is a typical pattern where the traditional…

Washback refers to the impact of a test on instruction and learning, with high-stakes tests exerting both positive and negative effects. This study examined the washback of an English exit exam (EEE) on English language learning at a Thai university where English-medium instruction is used in most academic disciplines. The EEE is an in-house…

State education agencies are sitting on rich repositories of quantitative and qualitative assessment data. This document is designed to provide a conceptual framework and implementation guidance that can help agency leadership leverage and interrogate student performance data in systematic ways for reporting, outreach, and planning purposes. The…

Background: One area that has been dramatically changed by artificial intelligence (AI) is educational environments. Chatbots, Recommender Systems, Adaptive Learning Systems and Large Language Models have been emerging as practical tools for facilitating learning. However, using such tools appropriately is challenging. In this regard, the…

Conducting a fair test is important for educational research. Unfair assessments can lead to gender disparities in academic achievement, ultimately resulting in disparities in opportunities, wages, and career choice. Differential Item Function [DIF] analysis is presented to provide evidence of whether the test is truly fair, where it does not harm…

Item response tree (IRTree) models are a flexible framework to control self-reported trait measurements for response styles. To this end, IRTree models decompose the responses to rating items into sub-decisions, which are assumed to be made on the basis of either the trait being measured or a response style, whereby the effects of such person…

Researchers and practitioners in the field of second language acquisition have come to realize the importance of non-cognitive skills such as self-efficacy and self-regulation in students' learning of a second language. However, there has been limited systematic research on such measures in the second language context and the validity and…

This study aimed to develop a comprehensive tool to assess underachievement in gifted students, incorporating input from parents, teachers, and students themselves. A total of 285 participants, including 95 gifted students, their parents, and teachers, were involved in the study. The results have revealed a four-factor structure for the Gifted…

Internalizing and externalizing problems are common targets for school mental health screening. Prior research supports the interpretation of scores from the Youth Internalizing Problems Screener (YIPS) and the Youth Externalizing Problems Screener (YEPS), which were developed separately yet intended as companion measures. We extended previous…

We describe the development and initial validation of the "ASL Fingerspelling and Number Comprehension Test" (ASL FaN-CT), a test of recognition proficiency for fingerspelled words in American Sign Language (ASL). Despite the relative frequency of fingerspelling in ASL discourse, learners commonly struggle to produce and perceive…

This study focused on developing physics test instruments for senior high school students on the topics of temperature and heat. The study aimed to determine (i) the quality of the test instrument content, (ii) the feasibility of the test instrument, and (iii) students' graphic representation abilities on "Temperature and Heat" topics.…

Mixed-format tests that include both multiple-choice (MC) and constructed-response (CR) items have become widely used in many large-scale assessments. When an item response theory (IRT) model is used to score a mixed-format test, the unidimensionality assumption may be violated if the CR items measure a different construct from that measured by MC…

Background: The Scenario Test is recognised for its effectiveness in assessing the interactive aspects of functional communication in people with post-stroke aphasia (PWA). Aims: To develop a Japanese version of the Scenario Test (Scenario Test-JP) and assess its reliability and validity. Methods & Procedures: Among 66 participants, we…

Student satisfaction in online live classes is considered an important criterion to evaluate the effectiveness of this instructional system. This study aims to develop a performance evaluation index to measure the satisfaction of students who have mastered Chinese language and literature through online live classes. Guided by survey techniques and…