The article reports three simulation studies conducted to find out whether the effect of a time limit for testing impairs model fit in investigations of structural validity, whether the representation of the assumed source of the effect prevents impairment of model fit and whether it is possible to identify and discriminate this method effect from…

Researchers are often inclined to test agreement or discrepancy hypotheses using difference scores. This commentary explains 2 mathematical-statistical principles underlying associations with difference scores and 2 conceptual-interpretation problems that make difference scores inappropriate for testing such hypotheses. The commentary provides…

In this paper, we address the following questions: What misconceptions do high school students exhibit in their first encounter with significance test problems through a repeated sampling approach? Which theory or framework could explain the presence and features of such patterns? With brief prior instruction on the use of Fathom software to…

Many large-scale educational surveys have moved from linear form design to multistage testing (MST) design. One advantage of MST is that it can provide more accurate latent trait [theta] estimates using fewer items than required by linear tests. However, MST generates incomplete response data by design; hence, questions remain as to how to…

This study investigates washback effects of IELTS test on university teachers' adoption of teaching materials in the classroom in China and see how a high-stake language exam affects their teaching practices and how they and their students perceive the effectiveness of the pedagogic adjustments. To this end, individual interview to five teacher…

Understanding the IELTS washback on learning in English as a Foreign Language (EFL) in higher education contexts remains limited. To fill in this research gap, this study investigated washback effects of the International English Language Testing System (hereafter called IELTS) as an English language proficiency exit test for final-year English…

Nearly all multidimensional models of communication competence have pragmatic competence at their core. Proper assessment of second language (L2) pragmatics makes many demands on L2 teachers, both in terms of understanding the construct and in language test development. Therefore, being assessment literate helps teachers in developing effective…

Reclassification is a critical threshold when English Learners (ELs) exit specialized language services and access all-English mainstream classrooms. Despite the mandates of the Every Student Succeeds Act, reclassification rates and time remain a pressing problem. A product of this malfunctioning system has been long-term ELs (LTELs). This article…

Construct validity and building validity arguments are some of the main challenges facing the language assessment community. The notion of construct validity and validity arguments arose from research in psychological assessment and developed into the gold standard of validation/validity research in language assessment. At a theoretical level,…

International large-scale assessments (ILSAs) are one of the most important tools policymakers and other educational stakeholders have to inform evidence based decision making for educational reform. Despite this, and the widespread use of ILSA data, results are sometimes misunderstood or misinterpreted. Here, we offer a brief guide to ILSAs and…

Documents are usually circulated as carriers of transparent information. They can serve as evidence of accountability. In fact, they embody the most desired value of managerialism, where the culture of audit and compliance is fully served and delivered in written and textual form. This article explores assessment by attending to its principal…

Current methods that simultaneously model differential testlet functioning (DTLF) and differential item functioning (DIF) constrain the variances of latent ability and testlet effects to be equal between the focal and the reference groups. Such a constraint can be stringent and unrealistic with real data. In this study, we propose a multigroup…

"The English Speaking Achievement Test for Japanese Junior High School Students" (ESAT-J) was introduced to contribute to levelling up public English education in Tokyo in 2022. Critics, however, have made claims in the mass media against the use of the test and stakeholder groups have called for its cancellation. This paper presents an…

The Elementary and Secondary Education Act (ESEA), as amended by Section 3113 of the Every Student Succeeds Act (ESSA), requires state educational agencies (SEAs) to establish and implement standardized, statewide entrance and exit procedures for English learners (ELs). WIDA provides the English language proficiency placement and annual…

The high-stakes College English Test (CET), developed, administered, and reformed over the last 20 years, has received great attention in the aspect of washback on teaching and learning from previous research. Very few studies explored its consequences in the workplace domain--being used as a screening lever. This research aimed to 1) compare…