This paper follows a line of logical argumentation to claim that what Samuel Messick conceptualized about construct validation has probably been misunderstood by some educational policy makers, practicing educators, and classroom teachers. It argues that, while Messick's unified theory of test validation aimed at (a) warning educational…

Construct validity and building validity arguments are some of the main challenges facing the language assessment community. The notion of construct validity and validity arguments arose from research in psychological assessment and developed into the gold standard of validation/validity research in language assessment. At a theoretical level,…

The purpose of this article of reflection is to raise awareness of how poor design of language assessments may have detrimental effects, if crucial qualities and technicalities of test design are not met. The article first discusses these central qualities for useful language assessments. Then, guidelines for creating listening assessments, as an…

National policy and standards documents, including the National Assessment of Educational Progress frameworks, the "Common Core State Standards" and the "Next Generation Science Standards," assert the need to assess critical-analytic thinking (CAT) across subject areas. However, assessment of CAT poses several challenges for…

This chapter introduces theory that undergirds the role of testing adaptations in assessment, provides examples of item modifications and testing accommodations, reviews research relevant to each, and introduces a new paradigm that incorporates opportunity to learn (OTL), academic enablers, testing adaptations, and inferences that can be made from…

Authentic and direct assessment of performance and products are examined in light of contrasting functions and purposes with implications for validation, especially those of specialized validity criteria for performance assessment. The roles of positive and negative consequences of validation are underscored, along with the need for evidence of…

Examines some problems of the Texas Assessment of Academic Skills (TAAS): multiple cutoff scores for passing the test and receiving a high school diploma, and artificially "tricky" items that disproportionately confuse language-minority students. Uses factor analysis and simulations to show ways to improve item selection for the TAAS.…

The idea that validity should be considered a property of inferences, rather than of assessments, has developed slowly over the past century. In early writings about the validity of educational assessments, validity was defined as a property of an assessment. The most common definition was that an assessment was valid to the extent that it…

This paper is rooted in an expanded view of construct validity, whereby the role of testers does not end in the development phase of the language tests they employ. Rather, testers need to follow the uses of these tests and examine issues of utility, relevance, ethics, and interpretation. The studies reported here focused on three national…

Four major problem areas inhibit the standardized assessment of critical thinking (CT): (1) content validity; (2) construct validity; (3) technical jargon; and (4) background knowledge. Practical examples of framing multiple-choice items for assessment are suggested. In the area of content validity, new agreement about the definition of CT now…

Second language tests claiming to assess communicative competence are widespread, despite the vague nature of the construct. Sociolinguistic or intercultural competence is gradually gaining attention in the classroom, but testing has not kept pace, partly because of difficulty in defining the related skills. An opinion is that speech act theory…

Standardized tests often produce results that are inaccurate, inconsistent, and biased against minority, female, and low-income students. Such tests shift control and authority into the hands of the unregulated testing industry and can undermine school achievement by narrowing the curriculum, frustrating teachers, and driving students out of…