As evidence-based interventions are scaled, fidelity of implementation, and thus effectiveness, often wanes. Validated fidelity measures can improve researchers' ability to attribute outcomes to the intervention and help practitioners feel more confident in implementing the intervention as intended. We aim to provide a model for the validation of…

The field of educational measurement has evolved considerably since the first doctoral programs were established. In response, programs have typically tacked on courses that address newly developed theories, methods, tools, and techniques. As our review of current programs evidences, this approach produces artificial distinctions among topics and…

In order to construct and evaluate the validity argument for the Objective Structured Assessment of Technical Skills (OSATS), based on Kane's framework, we conducted a systematic review. We searched MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, Web of Science, Scopus, and selected reference lists through February 2013. Working in duplicate, we selected…

Aims: To examine the quality of screening and assessment practices at some of the most highly regarded adolescent substance use treatment programs in the United States. Methods: Between March and September 2005, telephone surveys were administered to directors of highly regarded programs. Several different publications and databases were then used…

Local assessment systems are being marketed as formative, benchmark, predictive, and a host of other terms. Many so-called formative assessments are not at all similar to the types of assessments and strategies studied by Black and Wiliam (1998) but instead are interim assessments. In this article, we clarify the definition and uses of interim…

This manuscript describes the development and testing of an Internet-based cost survey that was designed by the authors for the National Assembly on School-Based Health Care (NASBHC) to capture the costs of school-based health programs. The intent of the survey was twofold. First, the survey was designed to collect comprehensive data on costs in a…

Assessments labeled as formative have been offered as a means to improve student achievement. But labels can be a powerful way to miscommunicate. For an assessment use to be appropriately labeled "formative," both empirical evidence and reasoned arguments must be offered to support the claim that improvements in student achievement can be linked…

The articles in this issue, written from the perspectives of academics, practitioners, and publishers, show that examining the consequences of assessment is an important, large, and difficult task. Collaborative action by assessment developers, users, and the educational measurement community is needed if progress is to be made. (SLD)

Provides an argument for incorporating consideration of consequences into validity theory that is grounded in the reflexive nature of social knowledge. It also calls for the consideration of evidence of validity based on the actual discourse surrounding the practices and products of testing. (SLD)

Discusses how online programs are using new strategies of testing, both to teach students and to prove that distance education works; however, some educators question their efficacy. (EV)

Presents a brief discussion of the use and misuse of psychometric tests as well as discriminatory practices. Presents a case of freedom of choice. Discusses the need for further research with particular emphasis on program evaluation. (RS)

In many school districts, the pressure to raise test scores has created overnight celebrity status for formative assessment. Its powers to raise student achievement have been touted, however, without attending to the research on which these claims were based. Sociocultural learning theory provides theoretical grounding for understanding how…

This Critical Issue Bibliography (CRIB) Sheet focuses on assessment. Assessment has been a major trend in higher education over the last 10 years, with the nature of the literature changing significantly over the years. This annotated bibliography groups assessment resources into these categories: (1) general resources; (2) institutional…

Misuses of null hypothesis significance testing are reviewed and alternative approaches are suggested for carrying out and reporting statistical tests that might be useful to program evaluators. Several themes, including the importance of respecting the magnitude of Type II errors and describing effect sizes in units stakeholders can understand,…

Formative and summative evaluations in education are compared, and appropriate uses of these methods in program evaluation are discussed. The main purpose of formative evaluation is to determine a level of mastery of a learning task, along with discovering any part of the task that was not mastered. In other words, formative evaluation focuses the…