This paper demonstrates that a meta-analysis technique applied to the Solomon Four-Group Design (SFGD) can fail to find significance even though an earlier "weaker" test may have found significance. The meta-analysis technique was promoted by Braver and Braver as the most powerful single test for analyzing data from an SFGD. They…

Sixteen tables are presented for critical values of the larger of two sample correlation coefficients from two independent samples, given the sample size and the value of the smaller correlation. These tables allow quick assessment of significance without requiring calculation of the test statistic. (SLD)

A generalized measure of association and an associated test of significance are presented for nominal independent variables in which any number or combination of interval, ordinal, or nominal dependent variables can be analyzed. A permutation test of significance is provided for the new measure. (SLD)

Different ways that dependent variables can cause multivariate significance are illustrated. It is argued that univariate tests should not be used as follow-up procedures to identify the variables that can contribute to multivariate significance. Methods that can be used to determine the contributions individual variables make to multivariate…

An Edgeworth approximation for accurate significance probabilities for the Wilcoxon two-sample test is substantially simplified. A method is developed that allows quick calculations of very accurate probabilities. Exact formulas are given for most of the remaining cases, and tables are presented comparing the new simplification to likely…

One source of bias, regression artifacts, in program evaluations that use regression discontinuity design is identified, and its importance is discussed. A method is suggested for its correction. Although use of this method does not solve the problem of selection bias, it does improve the reliability of nonexperimental methods. (SLD)

Large sample chi-square tests of the significance of the difference between two correlated kappas, weighted or unweighted, are derived. Cases are presented with one judge in common between the two kappas and no judge in common. An illustrative calculation is included. (Author/SLD)

Statistical significance is often misinterpreted to mean replicability or generalizability of results, although a statistically significant difference does not equal a reliable difference. Sample splitting procedures may be a more accurate way of estimating research result generalizability. This type of cross-validation involves randomly dividing…

The contribution of individual variables to overall multivariate significance in a multivariate analysis of variance (MANOVA) is investigated using a combination of canonical discriminant analysis and Roy-Bose simultaneous confidence intervals. Difficulties with this procedure are discussed, and its advantages are illustrated using examples based…

The Galois-lattice is a graphic method for representing all possible developmental phases to reach a given total knowledge via different partial knowledge structures. Algorithms are proposed for incorporating statistical analysis into the Galois-lattice method. A comparison to the Rasch model and application for educational research are given. (BS)

A new coefficient--psi-star--is presented for measuring agreement on bidirectional response continua. The coefficient is designed to measure agreement in larger square contingency tables with even numbers of rows and columns. Advantages of the procedure are discussed. (SLD)

Use of "special" orthonormal mean contrasts and mean contrast variances can help educational researchers interpret a wide variety of repeated measures data. Most statistical packages allow educational researchers to test for differences across repeated measures using both the univariate mixed model F test and a multivariate test.…

The asymptotic sampling distributions (means and variances) are presented for the signed and unsigned estimates for the Rasch model, two-parameter model, and the three-parameter model with fixed lower asymptotes. Applications for item-bias research are discussed. (SLD)

An overview of the state of the art in psychological research is presented, with an emphasis on the attention given to effect sizes. The acceptance of small effect sizes for biomedical research is contrasted with the rejection of similar effect sizes for psychological research. The Binomial Effect Size Display is used to depict the practical…

The mean and variance of the statistic psi-star, which is useful for measuring agreement between two raters and allows for tests of significance in the case of large samples, are considered. Results may also be applied for the "G" index of J. W. Holley and J. P. Guilford (1964). (SLD)