Three procedures for evaluating the replicability of descriptive discriminant analysis (DDA) results are discussed. The techniques include cross-validation, the jackknife, and the bootstrap. Discriminant analysis is a multivariate technique used when group membership or classification is the focus of the analysis. DDA is used to describe major…

When analyzing open-ended or categorical questions, many times responses are cross-classified by other categorical variables. The resulting contingency tables are then analyzed using the chi square test of independence. This procedure leads to multiple significance tests and provides no method to assess higher order interactions. Loglinear…

Repeated measures experimental designs, often referred to as "within-subjects" designs, offer researchers opportunities to study research effects while "controlling" for subjects. These designs offer greater statistical power relative to sample size. However, threats to internal validity such as carryover or practice effects…

The calculation of the F statistic for a one-factor analysis of variance (ANOVA) and the construction of an ANOVA tables are easily implemented on a spreadsheet. This paper describes how to compute the p-value (observed significance level) for a particular F statistic on a spreadsheet. Decision making on a spreadsheet and applications to the…

This paper explains the underlying assumptions of the sampling distribution and its role in significance testing. To compute statistical significance, estimates of population parameters must be obtained so that only one sampling distribution is defined. A sampling distribution is the underlying distribution of a statistic. Sampling distributions…

Contrary to popular opinion, significance testing does not inform the researcher of the likelihood of the replication of results from current research findings. Result replicability has been ignored by researchers because of an overreliance on significance testing. Several alternatives have been offered to provide the researcher with more…

Different ways that dependent variables can cause multivariate significance are illustrated. It is argued that univariate tests should not be used as follow-up procedures to identify the variables that can contribute to multivariate significance. Methods that can be used to determine the contributions individual variables make to multivariate…

The purposes of the present paper are to address the historical development of statistical significance testing and to briefly examine contemporary practices regarding such testing in the light of these historical origins. Precursors leading to the advent of statistical significance testing are examined as are more recent controversies surrounding…

Provides normative data about the distribution of one measure of similarity of factor loadings, the congruence coefficient, through a Monte Carlo Technique. Matching of "chance" factor patterns was done by the method of Tucker. Statistical tests of the results, based on similarities of the method to canonical and multiple correlation,…

Maximum likelihood estimates of the free parameters, and an asymptotic likelihood-ratio test, are given for the hypothesis that one or more elements of a covariance matric are zero, and/or that two or more of its elements are equal. (Author/RC)

The data of B. D. Layton and B. Turnbull's (1974) two extra sensory perception (ESP) experiments are used to illustrate Bayesian hypothesis tests that provide more useful information than is obtained from significance tests. (Editor)

Controversy over the proper place of significance testing within scientific methodology has continued for some time. The suggestion that effect sizes are more important than whether results are significant is presented. Effect size can be defined as an estimate of how much of the dependent variable is accounted for by the independent variables.…

The use of multiple comparisons in analysis of variance (ANOVA) is discussed. It is argued that experimentwise Type I error rate inflation can be serious and that its influences are often unnoticed in ANOVA applications. Both classical balanced omnibus and orthogonal planned contrast tests inflate experimentwise error to an identifiable maximum.…

Two strategies, derived from J. P. Schaffer (1986), were compared as tests of significance for a complete set of planned orthogonal contrasts. The procedures both maintain an experimentwise error rate at or below alpha, but differ in the manner in which they test the contrast with the largest observed difference. One approach proceeds directly to…

Generalizability or invariance procedures have been known for over three decades. Through the years, these procedures have not been widely discussed or employed. One reason for the lack of use is that most of the articles on invariance procedures have been mathematically oriented. The mathematical orientation of research articles and the lack of…