Practical advice on frequently asked questions dealing with research and evaluation methodology is presented as rules of thumb, with citations to the author's sources. A statement in the literature is considered a rule of thumb if it meets one of the following criteria: (1) it is specifically called a rule of thumb; (2) it contains numbers in…

Results are presented of a survey designed to elicit information from labs and centers on the types of development models that are currently being used in product development efforts and the evaluation strategies or models that have been found to be most compatible with them. The case is made that a great deal of research needs to be conducted on…

A re-analysis of evaluation data concerning the Emergency School Aid Act shows that findings of the original evaluators are open to multiple interpretations. (JKS)

The authors assess and illustrate the problems of incorporating an evaluative component into social programs using a number of examples from the field of alcohol studies. They outline a series of steps that are necessary for a resolution of the problem. (Author)

Comments on Bell's article "The Principles of Economics from Now until Then," responding to the author's two general premises as well as citing specific arguments. Concludes that Bell's premises are based on a small and unrepresentative number of textbooks. Many textbooks that present material according to the recommendations were…

The normative implications of applying significance tests in evaluation research are examined. The authors conclude that evaluators often make normative decisions, based on the traditional .05 significance level in studies with small samples. Additional reporting of the magnitude of impact, the significance level, and the power of the test is…

In most selection validation research, only the upper and lower tails of the criterion distribution are used, often yielding misleading or incorrect results. Provides formulas and tables which enable the researcher to account more accurately for the distribution of criterion within the middle range of population. (Author/RW)

Since Shapiro's introduction of Eye Movement Desensitization and Reprocessing (EMDR) in 1989, it has been a highly controversial therapeutic technique. Critical reviews of Shapiro's initial study have highlighted many methodological shortcomings in her work. And early empirical research that followed Shapiro's original study has been criticized…

An evaluation of open education was shown to produce misleading results due to probable regression phenomena. These questionable results are now spread throughout the literature of education, sociology, and psychology. Researchers are advised to review, not merely summarize, prior articles. (Author/PN)

This article describes the potential measurement problem in analyzing costs and benefits of vocational education so that policymakers and educators can distinguish between proper use and misuse of this evaluation technique. (Author/CT)

Reviews elements of research that need to be monitored by funding agencies and/or organizations carrying out research, and examines the problems and possibilities of research into the impact of research. Presents a case study of a project that focused on three strands of research undertaken at the University of Sheffield (United Kingdom). (Author)

It is argued that reports of evaluation activities often leave much to be desired because they fail to portray the real trial-and-error processes of the research study and do not report on the logic in use at the time of the inquiry. (SLD)

Sees important aspect of graduate education as learning how to evaluate quality and validity of research. Notes that study of families presents particular challenges. Examines three methods to teach students to critique family research. Sees all three as holding promise in teaching students how to evaluate family research and notes that all three…

Findings from empirical research on evaluation ethics indicate an apparent lack of consensus within the field concerning what constitutes an ethical issue, the frequent occurrence of problems in the later stages of evaluation projects, and the perceived ethical significance of the tendency of evaluators to be more responsive to some stakeholders…

Presents an overview of the concepts and methods employed by outcome researchers evaluating intervention effectiveness in child welfare, and the problems associated with these methods that make their use by agencies impractical. Discusses concepts of statistical significance, effect size, and the meaningfulness of effects. Emphasizes the need for…