A method of applying the discriminant function to correlated samples was discussed for the researcher interested in utilizing multivariate statistics to pre-post data. By finding the linear combination that maximizes differences in the groups, the t-statistic can be computed for correlated samples. (Author)

The power of a statistical test is, in part, a function of the reliability of the dependable variable being analyzed. The substitution of sigma square divided by the reliability coefficient for sigma is proposed. This enables the researcher to incorporate dependent variable reliability information when determining the sample size required for a…

Two well known directional (one-tailed) tests of significance, mean difference and correlation coefficient, are presented within the multiple linear regression framework. Adjustments on the computed probability level are indicated. The case for a directional interaction research hypothesis is defended. Conservative adjustments on the computed…

The primary purpose of the present study was to investigate empirically the effect of multiple hypothesis testing on error rates in factorial ANOVA designs under a variety of controlled conditions. The per comparison, per experiment, and experimentwise error rates were investigated for three hypothesis testing procedures. The specific conditions…

Basic to many psychological investigations is the question of agreement between observers who independently categorize people. Several recent studies have proposed measures of agreement when a set of nominal scale categories have been pre-defined and imposed on both observers. This study, in contrast, developes a measure of agreement for settings…

The present study was directed at determining the extent to which the Type I Error rate is affected by violations in the basic assumptions of the q statistic. Monte Carlo methods were employed, and a variety of departures from the assumptions were examined. (Author)

The strategy for investigating convergent and discriminant test validity, known as the multitrait-multimethod matrix, is investigated. A nonparametric significance testing procedure is suggested and demonstrated. (JKS)

Although the primary responsibility rests with the authors of articles reporting nonsignificant results to demonstrate the worth of the results by discussing the power of the tests, consumers should be prepared to conduct their own power analyses. This article demonstrates the use of power analysis for the interpretation of nonsignificant…

A computer program randomly sampled ordered pairs of scores from known populations that departed from bivariate normal form and calculated correlation coefficients from sample values. Hypotheses were tested (1) that population correlations are zero using the t statistic; and (2) that population correlations have non-zero values using the r to z…

The normative implications of applying significance tests in evaluation research are examined. The authors conclude that evaluators often make normative decisions, based on the traditional .05 significance level in studies with small samples. Additional reporting of the magnitude of impact, the significance level, and the power of the test is…

Many research-related classes in social sciences present probability as a unified approach based upon mathematical axioms, but neglect the diversity of various probability theories and their associated philosophical assumptions. Although currently the dominant statistical and probabilistic approach is the Fisherian tradition, the use of Fisherian…

Null hypothesis significance testing alone is not sufficient for program evaluation. To assess program impact adequately, effect sizes should be reported and interpreted in the context of similar or alternate programs. A popular effect size for the treatment-control group design has been the standardized mean difference, delta. Several estimators…