Since 2007, two taxonomies have been proposed to identify the components of evaluation practice that may be specified in an evaluation policy. Little is known, however, about how these taxonomies align with evaluation policies developed by philanthropic foundations. Through thematic analysis, this article first compares 12 foundation evaluation…

The reason for conducting the present case study was the problems the researchers encountered during data collection for another research project (Primary Study) entitled "The effects of virtual versus traditional flipped classes on EFL learners' grammar knowledge, self-regulation, and autonomy." Two online questionnaires were…

Effective assessment practice requires clearly defining and operationalizing terminology. We illustrate the importance of this practice by focusing on academic "undermatching"--when students enroll in colleges that are less academically selective than those for which they are academically prepared. Undermatching has been viewed as a…

Background: There has been rapid growth in value-added assessment of teachers to meet the widely supported policy goal of identifying the most effective and the most ineffective teachers in a school system. The former group is to be rewarded while the latter group is to be helped or fired for their poor performance. But, value-added approaches to…

Value-added "Progress" measures are to be introduced for all English schools in 2016 as "headline" measures of school performance. This move comes despite research highlighting high levels of instability in value-added measures and concerns about the omission of contextual variables in the planned measure. This article studies…

The Investigators studied scoring reliability of Candidates' ten day unit plans of instruction through prescribed action research projects, across three academic years. Scoring of the projects in year one provided opportunities for further refinement of the action research evaluation methods in year two. Across three terms in years one and two…

This experimental project investigated the reliability and validity of rubrics in assessment of students' written responses to a social science "writing prompt". The participants were asked to grade one of the two samples of writing assuming it was written by a graduate student. In fact both samples were prepared by the authors. The…

P. R. Killeen's (2005a) probability of replication ("p[subscript rep]") of an experimental result is the fiducial Bayesian predictive probability of finding a same-sign effect in a replication of an experiment. "p[subscript rep]" is now routinely reported in "Psychological Science" and has also begun to appear in…

The purpose of the recently proposed "p[subscript rep]" statistic is to estimate the probability of concurrence, that is, the probability that a replicate experiment yields an effect of the same sign (Killeen, 2005a). The influential journal "Psychological Science" endorses "p[subscript rep]" and recommends its use…

Four experienced stuttering researchers viewed videodisks of spontaneous speech from chronic stutterers and attempted to locate the precise onset and offset of individual stuttering events. Results showed interjudge disagreements that challenge the reliability and validity of onset and offset judgments. Highly agreed stuttering events were…

In this commentary, the authors discuss some of the issues regarding the use of diagnostic classification models that practitioners should keep in mind. In the authors experience, these issues are not as well known as they should be. The authors then provide recommendations on diagnostic scoring.

This paper forms part of an exploration of assessment on one part-time higher education (HE) course: an in-service, professional qualification for teachers and trainers in the learning and skills sector which is delivered on a franchise basis across a network of further education colleges in the north of England. This paper proposes that the…

The author suggests combining grades from component assessments to provide an overall student assessment. He explores the concept of reliability and concludes that the overall assessment will be reliable only if the number of grades used to report component achievements equals or exceeds the number used to report overall achievement. (Author/CH)

Briefly outlines problems associated with definition and official certification of suicide and reviews literature pertaining to validity and reliability of suicide statistics. Considers process of suicide certification as a test, estimating its sensitivity, specificity, and predictive value, using data from studies reviewed. (NB)

Treatment integrity (also referred to as "treatment fidelity," "intervention integrity," and "procedural reliability") is an important methodological concerning both research and practice because treatment integrity data are essential to making valid conclusions regarding treatment outcomes. Despite its relationship to validity, treatment…