This response aims to critically evaluate the methodology and aims of the meta-analytic review written by Maggin and Johnson (2014). The present authors systematically provide responses for each of the original criticisms and highlight concerns regarding Maggin and Johnson's methodology, while objectively describing the current state of evidence…

In this commentary, we examine Braverman's insights into the trade-offs between feasibility and rigor in evaluation measures and reject his assessment of the trade-off as a zero-sum game. We, argue that feasibility and policy salience are, like reliability and validity, intrinsic to the definition of a good measure. To reduce the tension between…

In critiquing our paper on "The literacy performance of ex-Reading Recovery students between two and four years following participation in the program: Is this intervention effective for students with early reading difficulties?", Schwartz argues that we have engaged in pursuing political and ideological agendas as part of our ongoing…

In this article, the authors responded to nine commentaries by 17 contributors to their article, "Issues in Differential Response." The authors found that a majority of the respondents agreed with the major conclusions of "Issues in Differential Response." However, there were varying degrees of disagreement regarding the…

Research syntheses are appealing because they enable decision makers to determine quickly whether policies, programs, and practices will have effects on student achievement and, if so, the magnitudes of the likely effects. Such syntheses should present objective, clear, scientifically accurate, and defensible evidence in terms that educators can…

Traditional methods for preparing systematic reviews and syntheses of effectiveness studies rely on a limited set of methodological criteria to include studies that measure and report effects too narrowly to forward the mission of evidence-based practice. This article discusses why and how the criteria for study selection, evidence screening, and…

"Black box" evaluation considers program input and outcomes (what) without considering process (how and why). A broader approach to evaluation based on implementation process theory explores the social system context of programs and provides information useful for improving them. (SK)

The author proposes using an action research model to diagnose, develop, implement, and evaluate a career development (CD) program that fits the needs and goals of the organization. Phases of a CD program development plan (data gathering, analysis and synthesis, feedback, program development, pilot implementation, and tracking) are examined. (CT)

Data from nine Michigan elementary, middle, and high schools highlight the challenges and common strategies found in the use of student assessment data to meet public demands for accountability. The information gathered from assessment tools varies by frequency and type. Educators have, historically, underutilized data to guide practice because…

When different individuals operating in different contexts implement similar innovations, they approach the various parts of the innovation in different ways. When assessed according to the research procedures used to study innovation implementation at the Research and Development Center for Teacher Education at the University of Texas at Austin,…

Problems and difficulties encountered in designing evaluation studies of educational projects in public education are reviewed, noting evaluation issues related to data collection, data analysis, and reporting. It is suggested that evaluation reports include qualitative and quantitative information and focus on descriptive methods of analyzing and…

The purpose of this report is to comment on practice in the use of existing data bases in program evaluation and school improvement and to explore directions of increased and improved use. The report's impetus is the role that existing data plays in current efforts in local school districts and the possibilities for the future. To a great extent,…

Some of the methodological problems in analyzing and reporting qualitative data are addressed, and some solutions based on recent experiences in conducting several qualitative studies are proposed. The following are some guidelines for implementing an intuitive approach to analyzing and reporting qualitative data: (1) use a team of evaluators; (2)…

The paper presents a model to assist administrators of special education programs for multiply handicapped children in making budgetary decisions and in evaluating therapy programs. The model's use requires that the efficiency and effectiveness of the various therapeutic services be measured and that the service needs of the multiply handicapped…

Problems related to the evaluation of rehabilitation agencies and rehabilitation education programs are reviewed. Questioned are the assumptions that even shoddy evaluation is better than nothing and that the consumer is one of the best sources of evaluation data. Corruption of data and selective or distorted reporting of results are discussed.…