The teaching of English as a foreign language in Indonesia has a long history, and it is always important to ask whether the assessment of the students' language skills has been valid and reliable. A screening of many articles in several prominent databases reveal that a number of evaluation studies have been done by Indonesian scholars in the…

This study reviews existing research on the use of large language models (LLMs) for automatic item generation (AIG). We performed a comprehensive literature search across seven research databases, selected studies based on predefined criteria, and summarized 60 relevant studies that employed LLMs in the AIG process. We identified the most commonly…

Purpose: The goal of the current study was to conduct a substantive validity review of four autism knowledge assessments with prior psychometric support (Gillespie-Lynch in J Autism and Dev Disord 45(8):2553-2566, 2015; Harrison in J Autism and Dev Disord 47(10):3281-3295, 2017; McClain in J Autism and Dev Disord 50(3):998-1006, 2020; McMahon…

The growing diversity among test takers in second or foreign language (L2) assessments makes the importance of fairness front and center. This systematic review aimed to examine how fairness in L2 assessments was evaluated through differential item functioning (DIF) analysis. A total of 83 articles from 27 journals were included in a systematic…

As the use of process data in large-scale educational assessments is becoming more common, it is clear that data on examinees' test-taking behaviors can illuminate their performance, and can have crucial ramifications concerning assessments' validity. A thorough review of the literature in the field may inform researchers and practitioners of…

As survey research in second language acquisition grows in popularity, the adherence to best practices associated with questionnaire quality is critical for a better understanding of factors that influence second language (L2) development. To ensure that a self-report scale targets the construct of interest and does it consistently and accurately,…

The New Ecological Paradigm (NEP) scale adapted for use with children (NEP-C) is one of the most frequently used measures of children's environmental beliefs. Though widely utilized, the limitations of the NEP-C instrument are often overlooked. Based on a systematic synthesis of existing literature examining the NEP-C, we argue that the scale…

In Comparative Judgement (CJ) exercises, examiners are asked to look at a selection of candidate scripts (with marks removed) and order them in terms of which they believe display the best quality. By including scripts from different examination sessions, the results of these exercises can be used to help with maintaining standards. Results from…

The "Rating Scale of Impairment" (RSI; Goldstein & Naglieri, 2016b) is a norm-referenced measure of functional impairment. The RSI measures impairment in six domains, as well as overall impairment, based in part on the International Classification of Functioning, Disability, and Health. Functional impairment, as defined by the ICF…

Many test developers try to ensure the content validity of their tests by having external experts review the items, e.g. in terms of relevance, difficulty, or clarity. Although this approach is widely accepted, a closer look reveals several pitfalls need to be avoided if experts' advice is to be truly helpful. The purpose of this paper is to…

Brief multiple-choice question workshops are a prevalent part of the faculty development landscape. But do they work? Studies have documented that faculty member-written multiple-choice questions (fMCQs) are frequently flawed and do not live up to quality standards. Poor fMCQs have real consequences for students beyond annoyance. Fourteen studies…

Adult learners are attracted to learning opportunities (e.g. course offers) which seem promising in terms of allowing them to match their choices to their own perceived predispositions. To find out more about their personal learning style, some adult learners may fill in a questionnaire designed by researchers who aim (and claim) to enable both…

The work at hand is part of a wider study the aim of which was to determine what kind of factors influence failures in pragmatic items of an online adaptive placement test taken by a group of 34 Spanish students in their first year at university. A preceding analysis (Carrió & Martín, 2016) showed the type of personal factors, such as lack of…

This paper details a systematic literature review identifying problems in extant research relating to teachers' attitudes toward reporting child sexual abuse and offers a model for new attitude scale development and testing. Scale development comprised a five-phase process grounded in contemporary attitude theories, including (a) developing the…

The authors conducted a qualitative meta-analysis of the research on assessment accommodations for students who are deaf or hard of hearing. There were 16 identified studies that analyzed the impact of factors related to student performance on academic assessments across different educational settings, content areas, and types of assessment…