Problems in selection of optimal designs in item-response theory (IRT) models are resolved through a sequential design procedure that is a modification of the D-optimality procedure proposed by Wynn (1970). This algorithm leads to consistent estimates, and the errors in selecting the abilities generally do not greatly affect optimality. (SLD)

There are two general methods of cross-validation: empirical estimation, and formula estimation. In choosing a specific cross-validation procedure, one should consider both costs (e.g., inefficient use of available data in estimating regression parameters) and benefits (e.g., accuracy in estimating population cross-validity). Empirical…

Simplicity and efficiency in design and estimation are all important in deciding on sampling strategies. A simple model is given and illustrated for four practical situations to show how a good sampling strategy should be selected. (Author)

Standard statistical methods are used to analyze data that is assumed to be collected using a simple random sampling scheme. These methods, however, tend to underestimate variance when the data is collected with a cluster design, which is often found in educational survey research. The purposes of this paper are to demonstrate how a cluster design…

In a complex sampling scheme it is desirable to select the primary sampling units (PSUs) without replacement to prevent duplications in the sample. Since the estimation of the sampling variances is more complicated when the PSUs are selected without replacement, L. Kish (1965) recommends that the variance be calculated using the formulas…

A general framework for conducting generalizability analyses is presented. Generalizability theory is extended to situations in which the objects of measurement are not persons but other factors, such as instructional objectives, stages of learning, and treatments. (Author/PN)

This study comparing momentary time sampling against the real time obtained with handheld computers confirmed laboratory findings that short-elementary time sampling estimates percentage time accurately for a wide range of behavior frequencies and durations, and suggested that observers using momentary time sampling in a natural setting are able…

Large scale surveys usually employ a complex sampling design and as a consequence, no standard methods for estimation of the standard errors associated with the estimates of population means are available. Resampling methods, such as jackknife or bootstrap, are often used, with reference to their properties of robustness and reduction of bias. A…

This paper shares the findings of an inquiry that evaluated 50 survey articles published in a refereed journal, "Educational Administration Quarterly," by examining the 53 sample designs reported in the articles. The paper presents a typology of the sample designs identified, discusses the problems of sample selection and estimation…

Replication is important to viable scientific inquiry; results that will not replicate or generalize are of very limited value. Statistical significance enables the researcher to reject or not reject the null hypothesis according to the sample results obtained, but statistical significance does not indicate the probability that results will be…

A generalized variance criterion is used for sequential sampling in the two-parameter item response theory model. Some principles are offered to enable the researcher to select the best sampling design for efficient estimation of item parameters. Topics include the choice of an optimality criterion, two-stage designs, and sequential designs. (SLD)

A study on sampling errors of variance components was conducted within the framework of generalizability theory by P. L. Smith (1978). The study used an intuitive approach for solving the problem of how to allocate the number of conditions to different facets in order to produce the most stable estimate of the universe score variance. Optimization…

Statistical power in music education was examined by taking an in-depth look at quantitative articles published in the "Journal of Research in Music Education" between 1987 and 1991, inclusive. Of the 109 articles of the period, 78 were quantitative, with both parametric and nonparametric procedures considered. Sample sizes were those…

This report describes a statistically designed experiment that was carried out in an operational setting to determine the contributions of different sources of variation to the unreliability of scoring. The experiment made novel use of partially balanced incomplete block designs that facilitated the unbiased estimation of certain main effects…

A method is proposed for calculating the sample size needed to achieve acceptable statistical power with a given test. The minimally important difference significant (MIDS) criterion for sample size is explained and supported with recommendations for determining sample size. The MIDS criterion is computationally simple and easy to explain. (SLD)