Since 2007, two taxonomies have been proposed to identify the components of evaluation practice that may be specified in an evaluation policy. Little is known, however, about how these taxonomies align with evaluation policies developed by philanthropic foundations. Through thematic analysis, this article first compares 12 foundation evaluation…

The reason for conducting the present case study was the problems the researchers encountered during data collection for another research project (Primary Study) entitled "The effects of virtual versus traditional flipped classes on EFL learners' grammar knowledge, self-regulation, and autonomy." Two online questionnaires were…

This experimental project investigated the reliability and validity of rubrics in assessment of students' written responses to a social science "writing prompt". The participants were asked to grade one of the two samples of writing assuming it was written by a graduate student. In fact both samples were prepared by the authors. The…

P. R. Killeen's (2005a) probability of replication ("p[subscript rep]") of an experimental result is the fiducial Bayesian predictive probability of finding a same-sign effect in a replication of an experiment. "p[subscript rep]" is now routinely reported in "Psychological Science" and has also begun to appear in…

In this commentary, the authors discuss some of the issues regarding the use of diagnostic classification models that practitioners should keep in mind. In the authors experience, these issues are not as well known as they should be. The authors then provide recommendations on diagnostic scoring.

This paper forms part of an exploration of assessment on one part-time higher education (HE) course: an in-service, professional qualification for teachers and trainers in the learning and skills sector which is delivered on a franchise basis across a network of further education colleges in the north of England. This paper proposes that the…

Briefly outlines problems associated with definition and official certification of suicide and reviews literature pertaining to validity and reliability of suicide statistics. Considers process of suicide certification as a test, estimating its sensitivity, specificity, and predictive value, using data from studies reviewed. (NB)

Treatment integrity (also referred to as "treatment fidelity," "intervention integrity," and "procedural reliability") is an important methodological concerning both research and practice because treatment integrity data are essential to making valid conclusions regarding treatment outcomes. Despite its relationship to validity, treatment…

This special series focused on treatment integrity in the child mental health and education field is timely. The articles do a laudable job of reviewing (a) the current status of treatment integrity research and measurement, (b) existing conceptual models of treatment integrity, and (c) the limitations of prior research. Overall, this thoughtful…

The ability of individuals to make accurate judgments about others is examined and literature on this subject is reviewed. A wide variety of situational factors affects the appraisal of performance. It is generally accepted that the purpose of the appraisal influences the accuracy of the appraiser. The instrumentation, or tools, available to the…

Research on student evaluation of college teachers' performance is briefly summarized. Lawrence M. Aleamoni offers four arguments in favor of student ratings: (1) students are the main source of information about the educational environment; (2) students are the most logical evaluators of student satisfaction and effectiveness of course elements;…

Four experienced stuttering researchers viewed videodisks of spontaneous speech from chronic stutterers and attempted to locate the precise onset and offset of individual stuttering events. Results showed interjudge disagreements that challenge the reliability and validity of onset and offset judgments. Highly agreed stuttering events were…

Discusses the need to evaluate the effectiveness of games and simulations that are used for training and educational purposes. Evaluation criteria are described, including validity, reliability, and utility; methods of measuring training device effectiveness are explained; and problems encountered with evaluations are discussed. (30 references)…

Reviews current educational software evaluation methods, highlights problems, and describes the York Educational Software Evaluation Scales (YESES), an alternative criterion based model. Panel evaluation used by YESES is explained and YESES results are compared with evaluations from the Educational Products Information Exchange (EPIE) to indicate…

Focuses upon three current issues in educational evaluation. Looks at the limitations of examinations and the interpretation of results in considering reliability and validity. Discusses grading and context of learning relative to formative and summative assessment. Deals with cultural background when exploring uniformity of testing versus…