The use of test scores as a performance measure in high-stakes educational accountability has become increasingly popular since the enactment of the No Child Left Behind Act of 2001 (NCLB), which imposed sanctions such as the threat of losing federal funds unless a state implemented a school accountability system that measures student progress…

The purpose of this statement is to inform those using or considering the use of value-added models (VAM) about their scientific and technical limitations in the evaluation of educators and programs that prepare teachers. The statement briefly reviews the background and current context of using VAM for evaluations, enumerates specific psychometric…

Performance standards are a powerful way to communicate K-12 student achievement (e.g., proficiency) and are the cornerstone of standards-based reform. As education reform shifts the focus to college and career readiness, approaches for setting performance standards need to be revised. We argue that the focus on assessing student readiness can…

The dangers of a single story in current U.S. large-scale writing assessment are that assessment practice does not align with theory and this practice has negative effects on instruction and students. In this article, I analyze the connections or lack of connections among writing theory, writing assessment, and writing instruction, critique the…

We describe an innovative automated test construction algorithm for building aligned achievement tests. By incorporating the algorithm into the test construction process, along with other test construction procedures for building reliable and unbiased assessments, the result is much more valid tests than result from current test construction…

Lissitz and Samuelsen (2007) have proposed an operational definition of "validity" that shifts many of the questions traditionally considered under validity to a separate category associated with the utility of test use. Operational definitions support inferences about how well people perform some kind of task or how they respond to some kind of…

Atkinson and Geiser (2009) make a strong argument for moving to a new form of college admissions testing using curriculum-based achievement tests. In making their case, however, they exaggerate the weaknesses of current tests such as the ACT and SAT by minimizing these tests' predictive utility and claiming a stronger relationship to socioeconomic…

Uses the process of model building in applied settings to examine design experiment (DE) methodology, noting that DE work presented in this issue does not discuss model fitting or broader models of validation. Discusses key areas for the DE community to address, positing that the concept of artifact failure in design research may be a more…

Responds to an article, "Guidelines for Quality in Autobiographical Forms of Self-Study," asserting that in addition to criteria for the quality of research, researchers must have ways to demonstrate its validity because of the moral and political aspects of educational research. The article's assertion is based in part on the existential nature…

Notes the need for researcher accountability in qualitative analysis, examining strategies used in working with doctoral students and offering suggestions for assessing and publicly disclosing the methodological rigor and analytical defensibility of qualitative research. Introduces tabular strategies for use in documenting the relationship between…

Proposes a systemically valid testing system that induces curricular and instructional changes in education systems to foster the development of the cognitive traits that tests are designed to measure. Analyzes test characteristics and outlines the principles of a systemically valid testing system. (FMW)

Scale-up is the practice of introducing proven interventions into new settings with the goal of producing similarly positive effects in larger, more diverse populations. Scale-up "research" examines factors that influence the effectiveness of interventions as they are brought to scale across settings. This article has three objectives. First, it…

Discusses the generalizability of critical thinking and the disposition to think critically. Argues that verbal reports of examinees' thinking on multiple-choice tests can explain the reasoning behind their answers and, thus, can be used to assess the inability to make credibility judgments. (FMW)

Describes tools for measuring content of instruction, content of instructional materials, and alignment between them, reporting findings about use of these tools and possible additional uses for research and practice. Asserts that the validity of data produced using these tools is good. Sketches an agenda for future work to improve the quality and…

Questions the claims by advocates of educational assessment reform, highlighting the work of Wiggins, who introduced authentic assessment. Discusses the origins of authentic tests, authenticity and validity, and assessment and educational philosophy, concluding that promotion of authentic assessment is flawed because "authentic" is misleading, and…