Becker (1967) poses the question "Whose side are we on?," a question which has become an enduring part of discussions within social scientific methodology. This paper explores the key issues in Becker's argument and considers its relevance to researchers today, locating this within a consideration of evaluation-based research and policy.…

The goal of this chapter is to provide a brief introduction to one of the most rigorous nonexperimental analytical methods currently employed by education researchers: regression discontinuity.

In the current article, we consider the influential position of the Programme for International Student Assessment (PISA) and discuss several methodological areas that demonstrate the need for caution when using and interpreting PISA results. We motivate our argument by briefly describing the program's increased influence in educational policy…

The authors encourage those in the field of school psychology to consider the use of learning disabilities assessment practices in relation to specific American Psychological Association and National Association of School Psychologists ethical codes and in regard to the American Educational Research Association, American Psychological Association,…

This "Instructor's Corner" describes a step forward on the journey to write, review, and publish high-quality qualitative research manuscripts. This article examines two existing perspectives on generating high-quality qualitative manuscripts and then compares and contrasts the different elements of each. First, an overview of Rocco's (2010) eight…

This study critically investigates the suitability of United Nations' composite indices and other related measures--among which the Gender Inequality Index just released in 2010--to capture gender inequalities in the context of "highly developed" countries, focusing on the case of Europe. Our results indicate that many of the gender gaps…

Written as four scenes that span a five-year period, this article examines the summative evaluation of an organized school reform programme in the United States from the teacher perspective. The study provides fine-grained details of how the evaluation of the particular reform effort went "awry" in the view of some teachers who directly…

A study by Mullen and Tallent-Runnels (2006) found significance in the differences between online and traditional students' reports of instructors' academic support, instructors' demands, and students' satisfaction. They also recognized that the limitation to their study was their demographic data. In an original report funded by the Alfred P.…

Although the Delphi is widely used, research on certain methodological issues is somewhat limited. After a brief introduction to the strengths, limitations, and methodological challenges of the technique, we share our experiences (as well as problems encountered) with an electronic Delphi of educational program evaluation (EPE) in the Asia-Pacific…

Rapidly declining response rates and the associated threat of nonresponse bias call into question the validity of data obtained through telephone surveys, a tool often used in evaluation. This article explores changes in nonresponse bias over time by examining three data points (1991, 1996, and 2002) from an annual household telephone survey…

Recommends that when repeated-measures Latin-square designs are used to counterbalance treatments across a procedural variable or to reduce the number of treatment combinations given to each participant, effects be analyzed statistically, and that in all uses, researchers consider alternative interpretations of the variance associated with the…

Considers relationship between statistically and clinically significant change. Sees Jacobson and Truax's index of clinically significant change as neglecting possible confounding of improvement rate estimates by regression to the mean. Describes alternative method (Edwards-Nunnally method) that incorporates an adjustment that minimizes this…

Conducted studies using two perspectives for obtaining evidence relevant to validity of autobiographical data. In Study 1, construct validity of standard psychological constructs (assertiveness, trustworthiness) obtained from autobiographies was equal to that of standard instruments designed to assess these constructs. Evidence for validity of…

Treatment integrity (also referred to as "treatment fidelity," "intervention integrity," and "procedural reliability") is an important methodological concerning both research and practice because treatment integrity data are essential to making valid conclusions regarding treatment outcomes. Despite its relationship to validity, treatment…

Reviewed and evaluated parent training research conducted between 1975 and 1990. Analyzed 148 published studies regarding effectiveness of parent training as intervention. Case studies reviewed exhibited several methodological shortcomings. Group studies reviewed also did not routinely collect treatment integrity information, and 50 percent of…