Automated scoring systems are typically evaluated by comparing the performance of a single automated rater item-by-item to human raters. This presents a challenge when the performance of multiple raters needs to be compared across multiple items. Rankings could depend on specifics of the ranking procedure; observed differences could be due to…

There is an increasing demand for assessments that can provide more fine-grained information about examinees. In response to the demand, diagnostic measurement provides students with feedback on their strengths and weaknesses on specific skills by classifying them into mastery or nonmastery attribute categories. These attributes often form a…

In this study, a multiple choice test which is composed of 19 articles which is prepared as per the scope of lesson of Measurement and Evaluation in Education, has been applied as interim exam to 207 teacher candidates who are getting education at the Faculty of Education. The difficulty levels of items which are in the test have been calculated…

In the study evaluated in this report, the authors asked what effect survey length has on student non-response rates to individual items on IDEA's "Diagnostic Feedback" (DF) and "Learning Essentials" (LE) forms. The approach was to analyze individual student ratings of classes contained in the 2015-2016 IDEA-CL database.…

Critical thinking skills are the 21st-century life skills that are needed by students. However, in elementary schools, there are no instruments that are truly effective and efficient to measure critical thinking skills. This research aims to develop an open-ended question assessment instrument to measure students' critical-thinking skills, to test…

The purpose of this study was to gauge the degree to which selected NWEA MAP Growth assessments are aligned to the Virginia Standards of Learning (SOL) and the extent to which MAP Growth reports can be used by school divisions to gauge student achievement relative to grade level and to identify learning gaps. The study was delimited to four MAP…

MOCCA is an online assessment of inferential reading comprehension for students in 3rd through 6th grades. It can be used to identify good readers and, for struggling readers, identify those who overly rely on either a Paraphrasing process or an Elaborating process when their comprehension is incorrect. Here a propensity to over-rely on…

The purpose of this study was to examine the presence of gender and ELL Differential Item Functioning (DIF) in a teacher-created mathematics benchmark test in one public charter school district in Northeast Georgia. DIF occurs when an item behaves differently in different subgroups, rather than measuring a test taker's true ability. The geometry…

This article examines the possible dependency of composite reliability on presentation format of the elements of a multi-item measuring instrument. Using empirical data and a recent method for interval estimation of group differences in reliability, we demonstrate that the reliability of an instrument need not be the same when polarity of the…

Continuously administered examination programs, particularly credentialing programs that require graduation from educational programs, often experience seasonality where distributions of examine ability may differ over time. Such seasonality may affect the quality of important statistical processes, such as item response theory (IRT) item…

Much attention is now given on producing quality graduates. Therefore, outcome-based education (OBE) in teaching and learning is now being implemented in Malaysia at all levels of education especially at higher education institutions. For implementing OBE, the design of curriculum and courses should be based on specified outcomes. Thus, the…

Examinees' thinking processes have become an increasingly important concern in testing. The responses processes aspect is a major component of validity, and contemporary tests increasingly involve specifications about the cognitive complexity of examinees' response processes. Yet, empirical research findings on examinees' cognitive processes are…

We explore whether the item characteristics of four-response multiple-choice questions with "paired" (non-independent) response options differ from those with unpaired response options in an operational licensure exam in terms of guessing behavior. Our analyses suggest that they do not. We discuss conditions where response pairing might…

The Partnership for Assessment of Readiness for College and Careers (PARCC) is a group of states working together to develop a set of assessments that measure whether students are on track to be successful in college and careers. Administrations of the PARCC assessment included three Prose Constructed Responses (PCR), one per task for English…

Elementary school students often exhibit a variety of conceptions associated with algebraic thinking that their teachers fail to recognize or understand. It is crucial that elementary school teachers possess knowledge of the variety of student conceptions and also have abilities to address varying states of conceptions. Otherwise, students who are…