Combining diverse methods in a single study raises a problem: What should be done when the findings of one method of investigation conflict with those of another? We illustrate this problem using an example in which three study phases--quantitative, qualitative, and intervention--were applied. The findings coming from the quantitative phase did…

The use of educational technology in K-12 classrooms has been gaining tremendous momentum across the country since the 1990s. Many school districts have been investing heavily in various types of technology, such as computers, mobile devices, internet access, and interactive whiteboards. Almost all public schools have access to the internet and…

Although the Delphi is widely used, research on certain methodological issues is somewhat limited. After a brief introduction to the strengths, limitations, and methodological challenges of the technique, we share our experiences (as well as problems encountered) with an electronic Delphi of educational program evaluation (EPE) in the Asia-Pacific…

This article summarizes critical evaluation needs, challenges, and lessons learned in translational research. Evaluation can play a key role in enhancing successful application of research-based programs and tools as well as informing program refinement and future research. Discussion centers on what is unique about evaluating programs and…

Researchers are urged to guard against presumption, and to focus on the issues gradually, as they emerge under observation. Naturalistic evaluation studies represent an interaction between a research viewpoint and a field activity. There is uniqueness to the research team's predilection, and uniqueness to the site. Progressive focusing requires…

This article presents the author's address at the 2007 "Journal of Applied Quantitative Methods" ("JAQM") prize awarding festivity. The festivity was included in the opening of the 4th International Conference on Applied Statistics, November 22, 2008, Bucharest, Romania. In the address, the author reflects on three theses that…

Studies of group differences in intelligence often invite conclusions about groups in general from studies of group differences in selected populations. The same design is used in the study of group differences in other traits as well. Investigators observe samples from two groups (e.g. men and women) in some accessible population, but seek to…

Receiver operating characteristic (ROC) analysis is being used increasingly to examine the memory processes underlying recognition memory. The authors discuss the methodological issues involved in conducting and analyzing ROC results, describe the various models that have been developed to account for these results, review the behavioral empirical…

The sheer number of instructional materials on the market creates a problem for educators because every new device must be carefully evaluated as to its feasibility before its successful use can be assured. (CK)

Classic parametric statistical significance tests, such as analysis of variance and least squares regression, are widely used by researchers in many disciplines, including psychology. For classic parametric tests to produce accurate results, the assumptions underlying them (e.g., normality and homoscedasticity) must be satisfied. These assumptions…

The present review examines research on the effects of technology use on reading achievement in K-12 classrooms. Unlike previous reviews, this review applies consistent inclusion standards to focus on studies that met high methodological standards. In addition, methodological and substantive features of the studies are investigated to examine the…

Aggregation of data is, by definition, an obscuring of details for the sake of achieving a summary. It is, therefore, potentially harmful to accuracy. Attention should be given to the aggregation that scores undergo prior to statistical tests. Two familiar research designs where this is important are a) in two groups/one measure cases, and b) in…

This report is a re-analysis of the data collected and analyzed by the Westinghouse Learning Corporation (WLC) and Ohio University concerning the average impact of Project Head Start on the cognitive development of a nationwide sample of children. The re-analysis was considered necessary because of unclear methodological issues in evaluation…

Identifies problems in evaluation research in organizational development (OD), including definition of objectives, research design, sampling, measurement, statistical validity and reliability, and interpretation of results. Recommends improvements in each evaluation area and suggests using utility theory, which requires estimation of all costs and…