We examined the feasibility and results of a multilevel multidimensional nominal response model (ML-MNRM) for measuring both substantive constructs and extreme response style (ERS) across countries. The ML-MNRM considers within-country clustering while allowing overall item slopes to vary across items and examination of whether certain items were…

Graphophonemic (GP) knowledge of a language represents the foundation of a good learning start point. In the English as a foreign language (EFL) context, many learners lack adequate GP knowledge of English, resulting in difficulty to master the language skills even at the tertiary level. Thus, this study investigated first-year university,…

Teachers' support concerning learners' metacognitive awareness (MA) is crucial in learning. Previous research has focused mainly on students' MA. Therefore, this study describes how subject teachers and special teachers (N = 1,045) perceive how they support learners' MA in general education and in vocational education and training. The main…

This article is a scale development study aimed at developing valid and reliable scales to understand English language teachers' knowledge about and attitudes towards plurilingualism and interculturalism. It is a descriptive survey model of quantitative research. During the scale development phase, literature review was conducted to develop the…

The main points of Sijtsma and Green and Yang in Educational Measurement: Issues and Practice (34, 4) are that reliability, internal consistency, and unidimensionality are distinct and that Cronbach's alpha may be problematic. Neither of these assertions are at odds with Davenport, Davison, Liou, and Love in the same issue. However, many authors…

This study was designed to investigate the adequacy of EFL learners' abilities in three major dimensions of digital literacy skills and whether self-assessments of competence were consistent with their actual performance. It also identified factors that affected learners' use of the selected digital literacy skills. To this end, 60 Saudi EFL…

Despite the majority of MCQ writing guides discouraging the use of negatively-worded multiple choice questions (NWQs), they continue to be regularly used both in locally produced examinations and commercially available questions. There are several reasons why the use of NWQs may prove resistant to sound pedagogical advice. Nevertheless, systematic…

Situational judgment tests (SJTs) are widely agreed to be a measurement technique. It is also widely agreed that SJTs are a questionable methodological choice for measurement of psychological constructs, such as behavioral competencies, due to a lack of evidence supporting appropriate factor structures and high internal consistencies.…

Large-scale assessments are supported by a large item pool. An important task in test development is to assign items into scales that measure different characteristics of individuals, and a popular approach is cluster analysis of items. Classical methods in cluster analysis, such as the hierarchical clustering, K-means method, and latent-class…

This article examines whether the way that PISA models item outcomes in mathematics affects the validity of its country rankings. As an alternative to PISA methodology a two-parameter model is applied to PISA mathematics item data from Canada and Finland for the year 2012. In the estimation procedure item difficulty and dispersion parameters are…

Assessing job applicants' general mental ability online poses psychometric challenges due to the necessity of having brief but accurate tests. Recent research (Myszkowski & Storme, 2018) suggests that recovering distractor information through Nested Logit Models (NLM; Suh & Bolt, 2010) increases the reliability of ability estimates in…

This study conducted a development and validation of a vignette-based grit scale (Grit-V). The purpose was to determine if the use of vignettes provided more validity evidence than short simple-sentence item endorsements. The data were gathered from a sample population of undergraduate students from The University of Alabama. Focus group data were…

This article discusses regression effects that are commonly observed in Angoff ratings where panelists tend to think that hard items are easier than they are and easy items are more difficult than they are in comparison to estimated item difficulties. Analyses of data from two credentialing exams illustrate these regression effects and the…

Substantial progress has been made toward applying technology enhanced conversation-based assessments (CBAs) to measure the English-language proficiency of English learners (ELs). CBAs are conversation-based systems that use conversations among computer-animated agents and a test taker. We expanded the design and capability of prior…

Effective listening comprehension skills are an important prerequisite for the academic success of primary school students. However, the assessment of listening skills in the instructional language appears to have received only scant attention in the literature. Therefore, the goal of the present study was twofold. Firstly, a comprehensive…