Randomized experiments are commonly used to evaluate the effectiveness of educational interventions. The goal of the present investigation is to develop small-sample corrections for multiple contrast hypothesis tests (i.e., F-tests) such as the omnibus test of meta-regression fit or a test for equality of three or more levels of a categorical…

This Monte Carlo study explored relationships among standard and unstandardized regression coefficients, structural coefficients, multiple R_ squared, and significance level of predictors for a variety of linear regression scenarios. Ten regression models with three predictors were included, and four conditions were varied that were expected to…

The degree to which reliability affects the power of analysis of variance (ANOVA) tests involving one factor with two and three samples was quantified and tabulated by taking into account sample size, level of significance, and true score effect size. Results confirm a substantial effect on power. (SLD)

Three strategies for augmenting the interpretation of significance test results are illustrated. Determining the most suitable indices to use in evaluating empirical results is a matter of considerable debate among researchers. Researchers increasingly recognize that significance tests are very limited in their potential to inform the…

An evaluation of the rules-of-thumb used to determine the minimum number of subjects required to conduct multiple regression analyses suggests that researchers who use a rule of thumb rather than power analyses trade simplicity of use for accuracy and specificity of response. Insufficient power is likely to result. (SLD)

This study investigated seven methods for analyzing multivariate group differences. Bonferroni t statistics, multivariate analysis of variance (MANOVA) followed by analysis of variance (ANOVA), and five other methods were studied using Monte Carlo methods. Methods were compared with respect to (1) experimentwise error rate; (2) power; (3) number…