This "What Works Clearinghouse Procedures and Standards Handbook (Version 3.0)" provides a detailed description of the standards and procedures of the What Works Clearinghouse (WWC). The remaining chapters of this Handbook are organized to take the reader through the basic steps that the WWC uses to develop a review protocol, identify…

This paper challenges Hanushek and Woessmann's (2008) contention that the quality and not the quantity of schooling determines a nation's rate of economic growth. I first show that their statistical analysis is flawed. I then show that when a nation's average test scores and average schooling attainment are included in a national income model,…

Classic parametric statistical significance tests, such as analysis of variance and least squares regression, are widely used by researchers in many disciplines, including psychology. For classic parametric tests to produce accurate results, the assumptions underlying them (e.g., normality and homoscedasticity) must be satisfied. These assumptions…

The normative implications of applying significance tests in evaluation research are examined. The authors conclude that evaluators often make normative decisions, based on the traditional .05 significance level in studies with small samples. Additional reporting of the magnitude of impact, the significance level, and the power of the test is…

Researchers increasingly recognize that significance tests are limited in their ability to inform scientific practice. Common errors in interpreting significance tests and three strategies for augmenting the interpretation of significance test results are illustrated. The first strategy for augmenting the interpretation of significance tests…

A novel solution to the statistical problems in an evaluation of rare events is described. The significance of variations in the number of child homicides is analyzed in a binary time series of "active" months for monitoring future incidence and related systemic events. (Author/CM)

Three criticisms of overreliance on results from statistical significance tests are noted. It is suggested that: (1) statistical significance tests are often tautological; (2) some uses can involve comparisons that are not completely sensible; and (3) using statistical significance tests to evaluate both methodological assumptions (e.g., the…

Presentation of quantitative research on the family often suffers from a tendency to interpret findings on a statistical rather than substantive basis. Advocates the use of data analysis that lends itself to an intuitive understanding of the nature of the findings, the strength of the association, and the import of the result. (JPS)

Ten articles on the use of multilevel data in educational research are presented. Topics include model specification and building, analytical procedures, aggregation bias, least squares statistics, hierarchical linear models, significance testing, and testing multilevel product-process networks. (TJH)

Reports on a review of empirical research reports submitted over the past several years to the "American Journal of Distance Education" that identified common analytic problems and errors often overlooked by distance education researchers. Topics discussed include significance testing; assessment issues, including reliability and…

Federal assistance for special educational programs makes necessary the regular study of evaluations of thousands of innovations in compensatory education, bilingual education, and reading programs. The results are reported to the President and to Congress. However, investigating organizations find only a few programs with adequate evidence and…

This paper details the origins of the media comparison study methodology, its current use as an evaluation instrument in distance education, and recommendations for more stringent discrimination between research and evaluation in the field of distance learning. The following topics are discussed: (1) the history of research in instructional…