This study evaluated the efficacy of two proposed methods in an operational standard-setting study conducted for a high-stakes language proficiency test of the U.S. government. The goal was to seek low-cost modifications to the existing Yes/No Angoff method to increase the validity and reliability of the recommended cut scores using a convergent…

Cognitive diagnostic models (CDMs) have received much interest within the field of language testing over the last decade due to their great potential to provide diagnostic feedback to all stakeholders and ultimately improve language teaching and learning. A large number of studies have demonstrated the application of CDMs on advanced large-scale…

Vocabulary assessment is an important part of measuring language proficiency in both monolingual and bilingual children. The LITMUS Cross-Linguistic Lexical Tasks (CLT) provides a framework for assessing the vocabulary of monolingual and bilingual children using a standardized procedure and comparable stimuli across languages. All language…

Panahi and Mohebbi review James Dean Brown's 50-years of research in language testing, curriculum development and research statistics with reference to an impressionistic framework for analysis containing two components with their subcomponents: Annotations (i.e., briefing and implications) and main concepts and themes (i.e., testing and teaching…

As diagnostic classification models become more widely used in large-scale operational assessments, we must give consideration to the methods for estimating and reporting reliability. Researchers must explore alternatives to traditional reliability methods that are consistent with the design, scoring, and reporting levels of diagnostic assessment…

This dissertation study explored the feasibility of using authenticated spoken texts to test L2 Chinese listening comprehension. The spoken texts used in the study were created using an "authenticating" technique, in which scripted spoken Chinese texts were infused with characteristics of real-world, unscripted spoken Chinese. In the…

The rising status of English as a global language prompts a call for a paradigm shift in English language teaching in order to correspond to the new sociolinguistic landscape. However, studies related to English language assessment, which can catalyze the transformation especially in a test-oriented context like Thailand, remain limited. This…

Nearly all multidimensional models of communication competence have pragmatic competence at their core. Proper assessment of second language (L2) pragmatics makes many demands on L2 teachers, both in terms of understanding the construct and in language test development. Therefore, being assessment literate helps teachers in developing effective…

Language proficiency diagnosis is essential to extract fine-grained information about the linguistic knowledge states and skill mastery levels of test takers based on their performance on language tests. Different from comprehensive standardized tests, many language learning apps often revolve around word-level questions. Therefore, knowledge…

This report is an update of the assessment principles and guidelines for English language learners published in 2013 (Thurlow, Liu, Ward, & Christensen). That report, which was developed by the Improving the Validity of Assessment Results for English Language Learners with Disabilities (IVARED) project, presented essential principles of…

Drawing on duoethnography, the teacher researchers in the present study interacted with the relevant literature, engaged in dialogs, and shared artifacts to examine their knowledge, beliefs, and practices regarding fairness and justice considerations in technology-enhanced language classroom assessment. Under the domain of knowledge, they…

Background/purpose: This study aims to analyze the need for local wisdom-based Indonesian as a Foreign Language (BIPA) evaluation materials for Thai students and propose a design to enhance language proficiency and cultural awareness. The growing interest in learning Indonesian stems from diverse purposes, including business, academics, and…

Researchers have used different measures to examine participants' second-language (L2) proficiency, yet it remains unclear how these different measures influence research results on the effect of L2 proficiency on the target variable. The present research explored the modulation effect of four different measures of L2 Chinese proficiency on…

Thinking skills play a critical role in determining language performance. Recent advancement in cognitive diagnostic modelling (CDM) provides a powerful tool for obtaining fine-grained information regarding these thinking skills during reading. Studies are scant, however, exploring the relations between thinking skills and language performance,…

In language performance tests, raters are important as their scoring decisions determine which aspects of performance the scores represent; however, raters are considered as one of the potential sources contributing to unwanted variability in scores (Davis, 2012). Although a great number of studies have been conducted to unpack how rater…