In this response, we first show that Simpson's proposed analysis answers a different and less interesting question than ours. We then justify the choice of prior for our Bayes factors calculations, but we also demonstrate that the substantive conclusions of our article are not substantially affected by varying this choice.

A recent paper uses Bayes factors to argue a large minority of rigorous, large-scale education RCTs are "uninformative." The definition of "uninformative" depends on the authors' hypothesis choices for calculating Bayes factors. These arguably overadjust for effect size inflation and involve a fixed prior distribution,…

Contradictory findings from "well-implemented" rigorous evaluations invite researchers to identify the differences that might explain the contradictions, helping to generate testable hypotheses for new research. This panel will examine efforts to ensure that the large number of local evaluations being conducted as part of four…

Prior learning assessment (PLA) is the process of evaluating learning, regardless of when or where the learning occurred, and, if the learning is at the college level, awarding college credit to the possessor of that learning. This article provides an overview of the PLA program at a large public U.S. institution and presents the findings of the…

This paper contains results of an empirical study that tested the efficacy and acceptability of two templates designed to fully involve students in proper and fair peer-assessments of their group project work (GPW) by providing concrete evidence of independent progressive documentation of their peers' contributions to the work-process and…

Studies that examine the impacts of education interventions on key student, teacher, and school outcomes typically collect data on large samples and on many outcomes. In analyzing these data, researchers typically conduct multiple hypothesis tests to address key impact evaluation questions. Tests are conducted to assess intervention effects for…

The assumption of equal treatment implementation is questioned. Through the reanalysis of data from a nutrition supplementation program evaluation, the power of the analysis of treatment effects is shown to increase when data on the level of treatment implementation is included. (Author/CM)

Misuses of null hypothesis significance testing are reviewed and alternative approaches are suggested for carrying out and reporting statistical tests that might be useful to program evaluators. Several themes, including the importance of respecting the magnitude of Type II errors and describing effect sizes in units stakeholders can understand,…

Designed for use with Research for Better Schools' evaluation system, this guide focuses upon basic concepts and statistical procedures applicable to specific research hypotheses emerging from evaluation questions. The first of three sections presents a discussion of descriptive statistics in terms of frequency, central tendency, variability, and…

Many program evaluations involve some type of statistical testing to verify that the program has succeeded in accomplishing initially established goals. In many cases, this takes the form of null hypothesis significance testing (NHST) with t-tests, analysis of variance, or some form of the general linear model. This paper contends that, at least…

Definitions of and approaches to naturalistic evaluation are discussed. The qualitative aspects of the discipline are reviewed; and use of naturalistic evaluation in descriptions, illustrations, explications, hypothesis testing, and assessment of public spending are defined. A new paradigm for naturalistic evaluation is proposed. (TJH)

This guide to program evaluation presents questions that illuminate aspects of the evaluation process. Issues to consider in program evaluation are: (1) the intended results of the program; (2) hypotheses about the possible results of the program; (3) the types of data that will indicate achievement of the intended results and the variables that…

The linked judgments approach is offered as a potential solution to problems of statistical conclusion validity when judges' ratings are employed in pretest-posttest designs. (Author/GK)

Provided are hypotheses and evaluation forms to evaluate vocational-occupational-technical education (V.O.T.E.) programs whose purposes include the mainstreaming of handicapped students. Materials are geared toward testing the following five hypotheses: most occupational administrators can cite specific examples of formal or informal evaluations…