This paper describes an on-going project to develop a formative, inferential reading comprehension assessment of causal story comprehension. It has three features to enhance classroom use: equated scale scores for progress monitoring within and across grades, a scale score to distinguish among low-scoring students based on patterns of mistakes,…

This article describes the results of Action Research conducted in an ESP classroom of Dhofar University located in Oman. Following the call of Oman Vision 2040 to emphasize educational practices that promote the development of higher-order cognitive processes, this study raises the following question: Can an online multiple choice question (MCQ)…

This research aimed to evaluate the students' conceptual understanding and to diagnose the students' preconceptions in elaborating the particle characteristics of matter by development of diagnostic instrument as well as Rasch model response pattern analysis approach. Data were acquired by 25 multiple-choice written test items distributed to 987…

The main purpose of this study is to investigate guessing in the Yes/No (YN) format vocabulary test. One-hundred-and-five university students took a YN test, a translation task and a multiple-choice vocabulary size test (MC VST). With matched lexical properties between the real words and the pseudowords, pseudowords could index guessing in the YN…

The current study sought to examine the validity of a General English Achievement Test (GEAT), administered to university students in the fall semester of 2018-2019 academic year, by hybridizing differential information (DIF) and differential distractor function (DDF) analytical models. Using a purposive sampling method, from the target population…

Automated summary evaluation is proposed as an alternative to rubrics and multiple-choice tests in knowledge assessment. Inbuilt rubric is a recent Latent Semantic Analysis (LSA) method that implements rubrics in an artificially-generated semantic space. It was compared with classical LSA's cosine-based methods assessing knowledge in a…

Integrating engineering into the K-12 science curriculum continues to be a focus in national reform efforts in science education. Although there is an increasing interest in research in and practice of integrating engineering in K-12 science education, to date only a few studies have focused on the development of an assessment tool to measure…

This research focuses on students' 5 hierarchical levels of Ordered Multiple-Choice (OMC) items for their extensive conceptualized understanding in the particulate nature of matter (PNM) chemistry. The basic framework for OMC items is to link students' conceptual understanding levels with possible cognitive responses. Developed as the substantial…

The purpose of this study was to develop the Critical Thinking Disposition Test in Biology as an alternative instrument in looking at the extent of one's disposition to critical thinking, especially in Biology University. Critical Thinking Disposition Tests in Biology are tests in the form of multiple choice based on biological cases. This…

Cognitive Diagnostic Assessment (CDA) is an alternative assessment which can give a clear picture of pupils' learning process and cognitive structures to education stakeholders so that appropriate instructional strategies can be designed to tailored pupils' needs. Coincide with this function, the Ordered Multiple-Choice (OMC) items were…

The calibration of test materials generally involves the interaction between empirical analysis and expert judgement. This paper explores the extent to which scale familiarity might affect expert judgement as a component of test validation in the calibration process. It forms part of a larger study that investigates the alignment of the…

In tests used to measure reading comprehension, validity is important in obtaining accurate results. Unfortunately, studies have shown that people can correctly answer some questions of these tests without reading the related passage. These findings bring forth the need to address whether this phenomenon is observed in multiple-choice only tests…

In multiple-choice tests, the quality of distractors may be more important than their number. We therefore examined the joint influence of distractor quality and quantity on test functioning by providing a sample of 5,793 participants with five parallel test sets consisting of items that differed in the number and quality of distractors.…

History education scholars have recognized the need for test validity research in recent years and have called for empirical studies that explore how to best measure historical thinking processes. The present study was designed to help answer this call and to provide a model that others can adapt to carry this line of research forward. It employed…

This study focused on the development and validation of a three-tier multiple-choice diagnostic instrument about the ecological footprint. Each question in the three-tier test comprised by; (a) the content tier, assessing content knowledge; (b) the reason tier, assessing explanatory knowledge; and (c) the confidence tier that differentiates lack…