The purpose of this study was to examine students' levels of engagement in educational practices by enrollment status, time of enrollment, and size of college in community colleges. Specifically, this study assessed the quality of the undergraduate education through students' self-reported data about their academic and nonacademic activities. The…

This report summarizes the findings from a national evaluation of mentoring programs funded under the U.S. Department of Education's Student Mentoring Program. The impact evaluation used an experimental design in which students were randomly assigned to a treatment or control group. Thirty-two purposively selected School Mentoring Programs and…

Despite the increasing criticism of statistical significance testing by researchers, particularly in the publication of the 1994 American Psychological Association's style manual, statistical significance test results are still popular in journal articles. For this reason, it remains important to understand the logic of inferential statistics. A…

This report examines both the substantiveness and statistical significance of differences between mean American College Testing (ACT) Program assessment scale scores. The first part of the report describes the development and results of a method for interpreting the substantiveness of mean differences in terms of test items correctly answered. For…

Three criticisms of overreliance on results from statistical significance tests are noted. It is suggested that: (1) statistical significance tests are often tautological; (2) some uses can involve comparisons that are not completely sensible; and (3) using statistical significance tests to evaluate both methodological assumptions (e.g., the…

This paper examines the problems caused by relying solely on statistical significance tests to interpret results in contemporary social science. The place of significance testing in educational research has often been debated. Among the problems in reporting statistical significance are questions of definition and terminology. Problems are also…

Discusses the necessity for considering significance level reversal when attempting to establish group equivalence prior to treatment. (GS)

Advantages of the use of multivariate commonality analysis are discussed and a small data set is used to illustrate the analysis and as a model to enable readers to conduct such an analysis. A noteworthy advantage of commonality analysis is that commonality honors the relationships among variables by determining the degree to which predictors in a…

This paper focuses on two methods of performing multiple comparisons (post hoc or unplanned versus a priori or planned) for determining exactly where statistically significant results arise when there are more than "k equals two" groups. Textbook views and recommendations regarding the use of these two types of multiple comparisons are…

R. V. Hopkins (1982) has criticized the use of means as the unit of analysis in situations where intact groups, such as classes, rather than individuals have been randomly assigned to various treatment conditions. Instead, Hopkins advocated the use of certain analysis of variance (ANOVA) models that, as far as test for treatment effects are…

The assumptions and consequences of applying conventional and newer statistical methods to meta-analytic data sets are reviewed. The application of the two approaches to a meta-analytic data set described by L. V. Hedges (1984) illustrates the differences. Hedges analyzed six studies of the effects of open education on student cooperation. The…

A growing controversy surrounds the strict interpretation of statistical significance tests in social research. Statistical significance tests fail in particular to provide estimates for the stability of research results. Methods that do provide such estimates are known as invariance or cross-validation procedures. Invariance analysis is largely…

The purpose of this paper is to demonstrate how multiple linear regression provides a viable statistical methodology for dealing with meta-analysis in general, and specifically with the issues of nonindependence and design complexity, such as multiple treatments. Since the F-test and t-test are special cases of the general linear model,…

The relationship between American College Testing Program (ACT) and California Test of Mental Maturity (CTMM) scores were explored. Four hundred and thirty-four undergraduate subjects of both sexes were selected from a midwestern university. Pearson product moment correlation (r) and Kendall rank correlation coefficient (tau) were used to measure…

Evaluation design is discussed in terms of conditions that an adult education intervention (product, practice) must meet to get Joint Dissemination and Review Panel (JDRP) approval. (Effectiveness, the sole criterion for JDRP approval, must be established by evaluation data adequate to tie the project and desired impact together in a…