Many scholars compared various item discrimination indices in real or simulated data. Item discrimination indices, such as item-total correlation, item-rest correlation, and IRT item discrimination parameter, provide information about individual differences among all participants. However, there are tests that aim to select a very limited number…

The goal of the present study was to address the analytical complexity of incorporating responses and response times through applying the Jeon and De Boeck mixture item response theory model in Mplus 8.7. Using both simulated and real data, we attempt to identify subgroups of responders that are rapid guessers or engage knowledge retrieval…

Questionnaire scales that are mixed-worded, i.e. include both positively and negatively worded items, often suffer from issues like low reliability and more complex latent structures than intended. Part of the problem might be that some responders fail to respond consistently to the mixed-worded items. We investigated the prevalence and impact of…

Given the significance of the test fairness, this study aimed to investigate a reading comprehension test for evidence of differential item functioning (DIF) based on English as a Foreign Language (EFL) learners' gender and their mode of learning (conventional vs. distance learning). To this end, 514 EFL learners were asked to take a 30-item…

Researchers often develop instruments using correctness scores (and a variety of theories and techniques, such as Item Response Theory) for validation and scoring. Less frequently, observations of children's strategies are incorporated into the design, development, and application of assessments. We conducted individual interviews of 833…

The present study scrutinized Iranian EFL learners' mode of learning (distance vs. conventional) as a probable source of bias in employing cognitive and metacognitive reading comprehension strategies. To this end, a total of 514 Iranian distance and conventional EFL learners were asked to take a 30-item multiple-choice reading comprehension test…

Purpose: The present study aimed to validate the Indonesian-language version of the psychological capital questionnaire (PCQ), specifically within the context of higher education, by utilising Rasch analysis to evaluate the reliability and validity aspect such as item-fit statistics, rating scale function, and differential item functioning of the…

The accurate assessment of analytical thinking in physics, particularly in magnetism, poses substantial challenges due to the limitations of conventional tools in measuring higher-order cognitive skills. This study aimed to validate an analytical skills test in physics, based on Bloom's Revised Taxonomy, with an emphasis on the dimensions of…

Many researchers use Cronbach's alpha to demonstrate internal consistency, even though it has been shown numerous times that Cronbach's alpha is not suitable for this. Because the intention of questionnaire and test constructers is to summarize the test by its overall sum score, we advocate summability, which we define as the proportion of total…

In typical differential item functioning (DIF) assessments, an item's DIF status is not influenced by its status in previous test administrations. An item that has shown DIF at multiple administrations may be treated the same way as an item that has shown DIF in only the most recent administration. Therefore, much useful information about the…

In most contexts of student skills assessment, whether the test material is administered by the teacher or within a learning environment, there is a strong incentive to minimize the number of questions or exercises administered in order to get an accurate assessment. This minimization objective can be framed as a Q-matrix design problem: given a…

We ask questions of students in order to probe their understanding. We design our questions in such a way that we can assess a student's progress towards an accurate worldview. However, there is a consensus that a performance gap exists in many physics assessments, where male students outperform their female peers. While early work in this area…

Evolutionary trees are central to learning about evolutionary processes, yet students at all educational levels struggle to read and interpret them. The synthetic tree-reading model (STREAM), based on published and not yet empirically tested models, was tested to determine whether the assumed hierarchy of the model could be substantiated and how…

In contrast with the assumptions made in standard measurement models used in large-scale assessments, students' performance may change during the test administration. This change can be modeled as a function of item position in case of a test booklet design with item-order manipulations. The present study used an explanatory item response theory…

This study presents results from a randomized experiment in the 2015-2017 National Survey of Family Growth, where a large national sample of U.S. individuals aged 15-49 was randomly assigned to one of two different versions of a survey question about sexual identity (one with three response options, including heterosexual, gay/lesbian, and…