Background: For decades, education researchers have relied on the work of Campbell, Cook, and Shadish to help guide their thinking about valid impact estimates in the social sciences (Campbell & Stanley, 1963; Shadish et al., 2002). The foundation of this work is the "validity typology" and its associated "threats to…

The "best available evidence" is one of the three basic inputs into evidence-based practice. This paper sets out a framework for evaluating the quality of systematic reviews that are intended to identify empirically supported interventions as a way of summarizing the best available evidence. The premise of this paper is that the process of…

The notion of validity in the social sciences is evolving and is influenced by philosophy of science, critiques of objectivity, and epistemological debates. Methodology for validation of the knowledge claims is diverse across different philosophies of science. In other words, definition and the way to establish of validity have evolved as…

The edTPA, a nationally available performance assessment for teacher candidates, has recently been developed and implemented in teacher education programs across the United States. Advocates make arguments for the need for such an assessment while critics of standardized performance assessments point out the dangers of standardization. This…

With funding from the Institute of Education Sciences (IES), the Virtual Performance Assessment project at the Harvard Graduate School of Education is developing and studying the feasibility of immersive virtual performance assessments (VPAs) to assess scientific inquiry of middle school students as a standardized component of an accountability…

As research on achievement goals has increased, so has the number of ways to measure goal orientation. In this article, we focus on the Patterns of Adaptive Learning Scales. A translated and adapted version of a subsection known as the Teacher Scales: Perceptions of the School Goal Structure for Students is examined in terms of factorial validity,…

The purpose of this article is to demonstrate application of a new framework, the validation framework (VF), to assist researchers in evaluating mixed research studies. Based on an earlier work by Dellinger and Leech, a description of the VF is delineated. Using the VF, three studies from education, health care, and counseling fields are…

The author responds to the recent work of Kaminski, Sloutsky, and Heckler (2008) and advances two major concerns about their research and its applicability to learning mathematics: a confounding variable that arises from the mathematical differences between the generic examples and concrete examples poses a threat to the construct validity of the…

This article presents the authors' response to the discussants of their article "A Suggested Change in Terminology and Emphasis Regarding Validity and Education" (this issue)--Susan E. Embretson, Joanna S. Gorin, Robert J. Mislevy, Pamela A. Moss, and Stephen G. Sireci. Their response is limited to a brief summarization of their position…

This article reports the first results of a long-term research project focusing on the detection and possible linguistic causes of differential item functioning (DIF) for second generation immigrant students in the Final Test of Primary Education in the Netherlands. The main aim of the project is to provide test constructors with information which…

This paper focuses on issues of data trustworthiness in praxis-oriented empirical work, research openly committed to the building of a world in which we can all flourish. The central argument is that those exploring the possibilities for a change-enhancing advocacy paradigm for doing empirical research in the human sciences must begin to be more…

The Rasch method, so often used in educational research, was used to analyze the construct validity of the criteria for premenstrual dysphoric disorder from "The Diagnostic and Statistical Manual of Mental Disorders." The method was used in contrast to using an external validation to establish the diagnostic utility of tests for…

In this article, Walt Haney, Michael Russell, and Damian Bebell summarize a decade of work using student drawings as a way to both document and change education and schooling. After a brief summary of more than one hundred years of literature on children's drawings, the authors point out that drawings have been little recognized as a medium of…

Construct validity has been an issue in assessment centers literature. This study developed a model for competency-based assessment centers by integrating competency modeling, training design, assessment centers, and multi-source feedback assessment. Several propositions in conjunction with the model were also developed to help resolve construct…

This study sought to develop a valid set of scales in the cognitive and affective domains for measuring the quality of life of university students. In addition the study attempted to illustrate the usefulness of Thomas Piazza's procedures for constructing valid scales in educational research. Piazza's method involves a multi-step construction of…