A small data set is used to show that correlations and standard deviations measured within an explicitly selected group need not be smaller than those within an applicant population. Both validity and reliability estimates within a selected group can exceed those within an applicant population. (Author/LMO)

Kelly's formula estimates sampling variance of correlation corrected for attenuation by using split-half reliabilities. In some cases, coefficient alpha estimate of reliability is preferable. A simulation study suggests a variation of Kelly's formula can be used appropriately with coefficient alpha. Kelly's formula is modified to accept…

Not infrequently, investigators assume that reliability for groups is greater than reliability for persons, or that the error variance for groups is less than that for persons. Using generalizability theory, it is shown that this "conventional wisdom" is not necessarily true. Examples are provided from the course-evaluation and the…

The average design effects for statistics estimated from the base-year National Longitudinal Study data are presented. Attempts to partition the effects into those due to stratification, clustering, and unequal weighting are discussed. The expected increases in subpopulation sample sizes due to oversampling are calculated and compared with the…

Longitudinal measures of change are the most important kinds of estimates being produced from the National Longitudinal Study of the High School Class of 1972 (NLS). Several topics are presented in this report, most of which build on previous variance components analyses. The major areas covered are effects of stratification and clustering,…

This report provides the statistical tools necessary to measure the extent of error that exists in organizational record data and group survey data. It is felt that traditional methods of measuring error are inappropriate or incomplete when applied to organizational groups, especially in studies of organizational change when the same variables are…

Statistical significance is misused in educational and psychological research when it is applied as a method to establish the reliability of research results. Other techniques have been developed which can be correctly utilized to establish the generalizability of findings. Methods that do provide such estimates are known as invariance or…

The stability of Rasch test item difficulty parameters was investigated under varying conditions. Data were taken from a mathematics achievement test administered to over 2,000 Australian students. The experiments included: (1) relative stability of the Rasch, traditional, and z-item difficulty parameters using different sample sizes and designs;…