Considers alternative strategies for optimizing statistical power in applied psychological research, including increasing sample size, combining predictors, and increasing alpha. Concluded that a priori power analysis should be a major consideration in any test of an hypothesis. Alpha level adjustment should be viewed as useful for increasing…

The research methodology literature in recent years has included a full frontal assault on statistical significance testing. An entire edition of "Experimental Education" explored this controversy. The purpose of this paper is to promote the position that while significance testing by itself may be flawed, it has not outlived its…

The general purpose of this study was to examine the efficiency of the Precision Efficacy Analysis for Regression (PEAR) method for choosing appropriate sample sizes in regression studies used for precision. The PEAR method, which is based on the algebraic manipulation of an accepted cross-validity formula, essentially uses an effect size to…

The primary objective of this study was to find the smallest sample size for which equating based on a random groups design could be expected to result in less overall equating error than had no equating been conducted. Mean, linear, and equipercentile equating methods were considered. Some of the analyses presented in this paper assumed that the…

The requirement of large sample sizes for calibrating items based on item response theory (IRT) models is not easily met in many practical pretesting situations. Although classical item statistics could be estimated with much smaller samples, the values may not be comparable across different groups of examinees. This study extended the authors'…

This study was designed to derive the distribution of a test statistic based on normal probability plots. The first purpose was to provide an empirical derivation of the critical values for the Line Test (LT) with an extensive computer simulation. The goal was to develop a test that is sensitive to a wide range of alternative distributions,…

Scores on the multiple-choice sections of alternate forms are equated through anchor-test equating for the Advanced Placement Program (AP) examinations. There is no linkage of free-response sections since different free-response items are given yearly. However, the free-response and multiple-choice sections are combined to produce a composite.…

The generalized graded unfolding model (GGUM) (J. Roberts, J. Donoghue, and J. Laughlin, 1998) is an item response theory model designed to analyze binary or graded responses that are based on a proximity relation. The purpose of this study was to assess conditions under which item parameter estimation accuracy increases or decreases, with special…

It is erroneous to extend or generalize the inter-rater reliability coefficient estimated from only a (small) proportion of the sample to the rest of the sample data where only one rater is used for scoring, although such generalization is often made implicitly in practice. It is shown that if inter-rater reliability estimate from part of a sample…

An alternative analysis of the two-group single response variable design is proposed. It involves the classification of experimental units to populations represented by the two groups. Three real data sets are provided to illustrate the utility of the classification analysis. A table of sample sizes required for the analysis is presented.…

Sample covariance matrices constructed with pairwise deletion for randomly missing data were used in a simulation with three sample sizes and five levels of missing data (up to 50%). Parameter estimates were unbiased, parameter variability was largely explicable, and no sample covariance matrices were nonpositive definite except for 50% missing…

Scales constructed using principal components and Rasch measurement methods are compared under conditions of unclear constructs and marginal sample sizes with three data sets of increasing complexity. Results of the two methods were identical when data were stable and the structure unidimensional. (SLD)

Studied the impact of systematic item feature variation on item statistical characteristics and the degree to which such information could be used as collateral information to supplement examinee performance data and reduce pretest sample size by generating 2 families of 48 word problem variants for the Graduate Record Examinations. Results with…

Introductory statistics texts contain inaccuracies in tables of critical values for Spearman's correlation. Paper provides table of critical values based on exact distribution for N greater than or equal to 3, less than or equal to 18; very accurate critical values for N greater than or equal to 19, less than or equal to 100, estimated using…

Exact testing was evaluated as a method for conducting Mantel-Haenszel differential item functioning (DIF) analyses with relatively small samples. A series of computer simulations found that the asymptotic Mantel-Haenszel and the exact method yielded very similar results across sample size, levels of DIF, and data sets. (SLD)