The remarkable strides in artificial intelligence (AI), exemplified by ChatGPT, have unveiled a wealth of opportunities and challenges in assessment. Applying cutting-edge large language models (LLMs) and generative AI to assessment holds great promise in boosting efficiency, mitigating bias, and facilitating customized evaluations. Conversely,…

Setting cut scores on (MSTs) is difficult, particularly when the test spans several grade levels, and the selection of items from MST panels must reflect the operational test specifications. In this study, we describe, illustrate, and evaluate three methods for mapping panelists' Angoff ratings into cut scores on the scale underlying an MST. The…

The Cross-Classified Mixed Effects Model (CCMEM) has been demonstrated to be a flexible framework for evaluating reliability by measurement specialists. Reliability can be estimated based on the variance components of the test scores. Built upon their accomplishment, this study extends the CCMEM to be used for evaluating validity evidence.…

Internationally, many assessment systems rely predominantly on human raters to score examinations. Arguably, this facilitates the assessment of multiple sophisticated educational constructs, strengthening assessment validity. It can introduce subjectivity into the scoring process, however, engendering threats to accuracy. The present objectives…

Reliability consists of both important social and scientific values and methods for evidencing those values, though in practice methods are often conflated with the values. With the two distinctly understood, a reliability argument can be made that articulates the particular reliability values most relevant to the particular measurement situation…

This study applied Messick's unified, multifaceted concept of construct validity to an electronic portfolio system used in a teacher education program. The subjects included 128 preservice teachers who recently completed their final portfolio reviews and student teaching experiences. Four of Messick's six facets of validity were investigated for…

Assessments labeled as formative have been offered as a means to improve student achievement. But labels can be a powerful way to miscommunicate. For an assessment use to be appropriately labeled "formative," both empirical evidence and reasoned arguments must be offered to support the claim that improvements in student achievement can be linked…

Considers what a responsible test developer would do to gain information to support the consequential basis of validity for a test early in the development. How the consequential basis of validity of the program would be monitored and reported during the life of the program is examined. The validity of the ACT Assessment is considered as if it…

The articles in this issue, written from the perspectives of academics, practitioners, and publishers, show that examining the consequences of assessment is an important, large, and difficult task. Collaborative action by assessment developers, users, and the educational measurement community is needed if progress is to be made. (SLD)

Provides an argument for incorporating consideration of consequences into validity theory that is grounded in the reflexive nature of social knowledge. It also calls for the consideration of evidence of validity based on the actual discourse surrounding the practices and products of testing. (SLD)

In many school districts, the pressure to raise test scores has created overnight celebrity status for formative assessment. Its powers to raise student achievement have been touted, however, without attending to the research on which these claims were based. Sociocultural learning theory provides theoretical grounding for understanding how…

Explores methods of assessing opportunity to learn (OTL) and presents data collected as part of a pilot study of an eighth-grade statewide mathematics assessment to explore issues of validity. Investigates the integrity of various dimensions thought to constitute OTL, analyzes the relationships among teachers' and students' self-reports, and draws…