The current paper intends to exploit the Many Facet Rasch Model to investigate and compare the impact of situations (items) and raters on test takers' performance on the Written Discourse Completion Test (WDCT) and Discourse Self-Assessment Tests (DSAT). In this study, the participants were 110 English as a Foreign Language (EFL) students at…

This study aimed to propose a new method for scoring C-Tests as measures of general language proficiency. In this approach, the unit of analysis is sentences rather than gaps or passages. That is, the gaps correctly reformulated in each sentence were aggregated as sentence score, and then each sentence was entered into the analysis as a polytomous…

Local item dependency (LID) occurs when test-takers' responses to one test item are affected by their responses to another. It can be problematic if it causes inflated reliability estimates or distorted person and item measures. The cued-recall reading comprehension test in Hu and Nation's (2000) well-known and influential coverage--comprehension…

Automated assessment using Natural Language Processing (NLP) has the potential to make English speaking assessments more reliable, authentic, and accessible. Yet without careful examination, NLP may exacerbate social prejudices based on gender or native language (L1). Current NLP-based assessments are prone to such biases, yet research and…

This paper explores differences in 437 learners' "foreign language classroom anxiety" (FLCA) in in-person and online English as a Foreign Language (EFL) classes before the outbreak of the pandemic and during the first lockdown in spring 2020. Statistical analyses of data gathered with a web survey revealed a slight, yet significant drop…

This study presents the validation process of a listening test based on a communicative language test proposed by Bachman (Fundamental considerations in language testing, 1990). It was administered to third-grade high school students by the sixteen Korean Provincial Offices of Education for Curriculum and Evaluation in September 2012 to assess…

This article examines the performance of heritage speakers on a bimodal acceptability judgment task that targeted morphologically complex words. A major goal of the study was to compare participants' acceptance of conventional and creative words. Data were collected from 57 adult heritage speakers of Spanish who were subsequently divided into two…

This study examines motivation and multilingualism using quantitative data from 468 language learners in China. More precisely, the point of inquiry was to investigate the language-specificity of the psychological aspects of self within the L2 Motivational Self System (L2MSS). The Exploratory Factor Analysis (EFA) with the L2 English data revealed…

The main objective of the article is to compare several methods of evaluating multiple-choice options through classical item analysis. The methods subjected to examination include the tabulation of choice distribution, the interpretation of trace lines, the point-biserial correlation, the categorical analysis of trace lines, and the investigation…

The main purpose of this quantitative-qualitative content analysis study was to compare IELTS and TOEFL listening and reading tests based on the representation of the learning objectives of Revised Bloom's taxonomy. To this end, 12 Academic IELTS listening and reading tests and 12 TOEFL iBT listening and reading tests were analyzed qualitatively…

This study is a conceptual replication of Rogers and Cheung's (2018) investigation into distribution of practice effects on the learning of L2 vocabulary in child EFL classrooms in Hong Kong. Following a pretest, treatment, delayed posttest design, 66 primary school students (Cantonese L1) studied 20 vocabulary items over three training episodes…

It is generally accepted that collocation is essential for establishing synonymy. They are related linguistic features which are important and useful for language learners. Given this importance, this present study considered the experience of 120 Thai English as a Foreign Language (EFL) students. The study explored the students' knowledge of…

This study examines the relationships among various major factors that may affect the difficulty level of language tests in an attempt to enhance the robustness of item difficulty estimation, which constitutes a crucial factor ensuring the equivalency of high-stakes tests. The observed difficulties of the reading and listening sections of two EFL…

The calibration of test materials generally involves the interaction between empirical analysis and expert judgement. This paper explores the extent to which scale familiarity might affect expert judgement as a component of test validation in the calibration process. It forms part of a larger study that investigates the alignment of the…

Technology-enhanced items (TEIs) are innovative, computer-delivered test items that allow test takers to better interact with the test environment compared to traditional multiple-choice items (MCIs). The interactive nature of TEIs offer improved construct coverage compared with MCIs but little research exists regarding students' performance on…