Basic to many psychological investigations is the question of agreement between observers who independently categorize people. Several recent studies have proposed measures of agreement when a set of nominal scale categories have been pre-defined and imposed on both observers. This study, in contrast, developes a measure of agreement for settings…

The present study was directed at determining the extent to which the Type I Error rate is affected by violations in the basic assumptions of the q statistic. Monte Carlo methods were employed, and a variety of departures from the assumptions were examined. (Author)

The strategy for investigating convergent and discriminant test validity, known as the multitrait-multimethod matrix, is investigated. A nonparametric significance testing procedure is suggested and demonstrated. (JKS)

A test of significance for identifying individuals who are most influenced by an experimental treatment as measured by pre-post test change score is presented. The technique requires true difference scores, the reliability of obtained differences, and their standard error of measurement. (Author/JKS)

Although the primary responsibility rests with the authors of articles reporting nonsignificant results to demonstrate the worth of the results by discussing the power of the tests, consumers should be prepared to conduct their own power analyses. This article demonstrates the use of power analysis for the interpretation of nonsignificant…

A computer program randomly sampled ordered pairs of scores from known populations that departed from bivariate normal form and calculated correlation coefficients from sample values. Hypotheses were tested (1) that population correlations are zero using the t statistic; and (2) that population correlations have non-zero values using the r to z…

A review of articles that report study results based on couple samples indicated response rates are rarely high enough for statistical inference. Four procedures that can be used to compensate for insufficient response rates (collecting information from nonparticipants, census comparisons, adjustment in analysis, and replication) are examined.…

The normative implications of applying significance tests in evaluation research are examined. The authors conclude that evaluators often make normative decisions, based on the traditional .05 significance level in studies with small samples. Additional reporting of the magnitude of impact, the significance level, and the power of the test is…

Many research-related classes in social sciences present probability as a unified approach based upon mathematical axioms, but neglect the diversity of various probability theories and their associated philosophical assumptions. Although currently the dominant statistical and probabilistic approach is the Fisherian tradition, the use of Fisherian…

Null hypothesis significance testing alone is not sufficient for program evaluation. To assess program impact adequately, effect sizes should be reported and interpreted in the context of similar or alternate programs. A popular effect size for the treatment-control group design has been the standardized mean difference, delta. Several estimators…

The probabilities of attaining varying magnitudes of standardized effect sizes by chance and when protected by a 0.05 level statistical test were studied. Monte Carlo procedures were used to generate standardized effect sizes in a one-way analysis of variance situation with 2 through 5, 6, 8, and 10 groups with selected sample sizes from 5 to 500.…

Reviews a recent paper by Levy (1982) which concluded that a new format of the Peabody Picture Vocabulary Test was valid for autistic children. Claims the analysis was inappropriate and presents a statistical test to compare differences between independent mean differences. Includes a reply by Stine Levy. (Author/JAC)