The Council for the Accreditation of Educator Preparation (CAEP) requires faculty from educator preparation programs to provide evidence of credential candidates' impact on K-12 student learning. However, there is a paucity of information on preparation programs' use of direct assessments of student learning to gauge credential candidate…

Recent meta-analyses of drug-court studies recognized the poor methodological quality of the evaluations, with only a few being randomized. This article critiques the design of the randomized studies from a statistical perspective. Learning points are identified for future drug-court studies and are applicable to evaluations both of other…

The article grounds the structure of the external evaluation of colleges at the level of the non-university higher education sector, and it reveals the problems met in this process as well as possibilities for their solution: the phenomenon of the external evaluation of colleges is discussed and its concept is presented; the external evaluation…

Good evaluation requires that evaluation efforts themselves be evaluated. Many things can and often do go wrong in evaluation work. Accordingly, it is necessary to check evaluations for problems such as bias, technical error, administrative difficulties, and misuse. Such checks are needed both to improve ongoing evaluation activities and to assess…

Lecoutre, Lecoutre, and Poitevineau (2010) have provided sophisticated grounding for "p[subscript rep]." Computing it precisely appears, fortunately, no more difficult than doing so approximately. Their analysis will help move predictive inference into the mainstream. Iverson, Wagenmakers, and Lee (2010) have also validated…

There is an increasing interest in the field of experiential education to move beyond simply documenting the value of experiential education programs and, instead, develop more evidence-based models for experiential education practice (cf., Gass, 2005; Henderson, 2004). Due in part to the diversity of experiential education programs, participants,…

P. R. Killeen's (2005a) probability of replication ("p[subscript rep]") of an experimental result is the fiducial Bayesian predictive probability of finding a same-sign effect in a replication of an experiment. "p[subscript rep]" is now routinely reported in "Psychological Science" and has also begun to appear in…

The sense that replicability is an important aspect of empirical science led Killeen (2005a) to define "p[subscript rep]," the probability that a replication will result in an outcome in the same direction as that found in a current experiment. Since then, several authors have praised and criticized 'p[subscript rep]," culminating…

The gold standard in making causal inference on program effects is a randomized trial. Most randomization designs in education randomize classrooms or schools rather than individual students. Such "clustered randomization" designs have one principal drawback: They tend to have limited statistical power or precision. This study aims to…

This is a critical review of methodological issues in the evaluation of adult literacy education programs in the United States. It addresses the key research questions: What are the appropriate methods for evaluating these programs under given circumstances. It identifies 15 evaluation studies that are representative of a range of adult literacy…

Develops random-effects probit model for case in which outcome of interest is series of correlated binary responses, obtained as product of longitudinal response process where individual is repeatedly classified on binary outcome variable or in multilevel or clustered problems in which individuals within groups are considered to share…

The purpose of this article is to extend the discussion of issues currently being debated on the need for more rigorous program evaluation in educational and other sectors of research, to the field of international development evaluation, reviewing the different approaches which can be adopted to rigorous evaluation methodology and their…

Course evaluation is integral to medical education. We discuss (1) distinctive features of medical education that impact on course evaluation, (2) a framework for course evaluations, (3) details that shape the evaluation process, (4) key measurement issues important to data gathering and interpretation, and (5) opportunities for expanding the…

Notes inconsistencies in drug abuse prevention research findings related to such issues as study design and methodology. Presents consensus recommendations made by drug abuse prevention researchers and practitioners who met at National Institute on Drug Abuse in 1989. Includes specific recommendations directed to modifying prevention approaches;…

In this pilot study, 14 adolescents kept diaries for 6 consecutive weeks during their pregnancies. Diary entries were analyzed for affective tone, emotional lability, and contextuality. Findings question psychometric properties of data gathered using one time, self-report measures with pregnant adolescents because of their fluctuating mood states.…