Large-scale educational assessments are sometimes considered low-stakes, increasing the possibility of confounding true performance level with low motivation. These concerns are amplified in remote testing conditions. To remove the effects of low effort levels in responses observed in remote low-stakes testing, several motivation filtering methods…

We explored three approaches to resolving or re-scoring constructed-response items in mixed-format assessments: rater agreement, person fit, and targeted double scoring (TDS). We used a simulation study to consider how the three approaches impact the psychometric properties of student achievement estimates, with an emphasis on person fit. We found…

The 2025-2026 Indiana Assessments Policy Manual communicates established guidelines regarding appropriate test administration in Indiana for key stakeholders including educators and Test Coordinators. This document contains policy guidance and appendices that delineate specific aspects of test implementation, including test security protocol,…

What is assessment literacy? It is a handful of fundamental understandings about the testing concepts and procedures that influence educational decisions. And it just might be the most cost-effective means of real school improvement. With characteristic humor and aplomb, assessment expert W. James Popham strips away the psychometrician-speak and…

The Nebraska Student-Centered Assessment System (NSCAS) is a statewide assessment system that embodies Nebraska's holistic view of students and helps them prepare for success in postsecondary education, career, and civic life. It uses multiple measures throughout the year to provide educators and decision-makers at all levels with the insights…

In this paper, we provide an overview of psychometric procedures and guidelines Educational Testing Service (ETS) uses to evaluate automated essay scoring for operational use. We briefly describe the e-rater system, the procedures and criteria used to evaluate e-rater, implications for a range of potential uses of e-rater, and directions for…

e-Learning plays an undoubtedly important role in today's education and assessment is one of the most essential parts of any instruction-based learning process. Assessment is a common way to evaluate a student's knowledge regarding the concepts related to learning objectives. In this paper, a new method for assessing the free text answers of…

Automated Essay Scoring (AES) has garnered a great deal of attention from the rhetoric and composition/writing studies community since the Educational Testing Service began using e-rater[R] and the "Criterion"[R] Online Writing Evaluation Service as products in scoring writing tests, and most of the responses have been negative. While the…

How to compute and report subscores for a test that was originally designed for reporting scores on a unidimensional scale has been a topic of interest in recent years. In the research reported here, we describe an application of multidimensional item response theory to identify a subscore structure in a test designed for reporting results using a…

Across the globe, educational tests are being used at a rapidly increasing rate. More recently, educational tests are being used to inform educational policy and for holding educators accountable for student learning. One reason educational assessments are used for these important purposes is that they are considered to provide reliable and…

The purpose of the study was to investigate the extent to which raters coming from diverse backgrounds exhibited different levels of rating ability while scoring speaking performances. The study also aimed to examine how raters with different backgrounds could develop their rating ability over time. For this purpose, raters' background…

At the heart of most assessments lies a set of questions, and those who write them must achieve "two" things. Not only must they ensure that each question elicits the kind of performance that shows how "good" pupils are at the subject, but they must also ensure that each mark scheme gives more marks to those who are…

Don't settle for assessing recall and comprehension only when you can use this guide to create assessments for higher-order thinking skills. Assessment expert Susan M. Brookhart brings you up to speed on how to develop and use test questions and other assessments that reveal how well your students can analyze, reason, solve problems, and think…

With the focus on how a sample of 25 Arab ESL learners respond to the TOEFL-iBT reading tasks, this study aimed to find out what strategies respondents tend to use, investigate if there are differences between high- and low-scorers in strategy use, and determine aspects of effective strategy use among respondents. Data were collected using a…

The purpose of this study was to determine the statistical difference of a proficient score on the third and forth grade teacher-created criterion-referenced math assessments, aligned to state standards, on the Tennessee Comprehensive Achievement Program (TCAP). This study examined if this statistical difference relates to minority,…