Background: Over the past decade, research on the development and assessment of tools to improve the generalizability of experimental findings has grown extensively (Tipton & Olsen, 2018). However, many experimental studies in education are based on small samples, which may include 30-70 schools while inference populations to which…

We consider Bayesian estimation of a hierarchical linear model (HLM) from small sample sizes. The continuous response Y and covariates C are partially observed and assumed missing at random. With C having linear effects, the HLM may be efficiently estimated by available methods. When C includes cluster-level covariates having interactive or other…

The unstructured multiple-attempt (MA) item response data in virtual learning environments (VLEs) are often from student-selected assessment data sets, which include missing data, single-attempt responses, multiple-attempt responses, and unknown growth ability across attempts, leading to a complex and complicated scenario for using this kind of…

Small samples are common in growth models due to financial and logistical difficulties of following people longitudinally. For similar reasons, longitudinal studies often contain missing data. Though full information maximum likelihood (FIML) is popular to accommodate missing data, the limited number of studies in this area have found that FIML…

Evaluators of education interventions are increasingly designing studies to detect impacts much smaller than the 0.20 standard deviations that Cohen (1988) characterized as "small." While the need to detect smaller impacts is based on compelling arguments that such impacts are substantively meaningful, the drive to detect smaller impacts…

Utilizing planned missing data (PMD) designs (ex. 3-form surveys) enables researchers to ask participants fewer questions during the data collection process. An important question, however, is just how few participants are needed to effectively employ planned missing data designs in research studies. This article explores this question by using…

Educational researchers commonly use the rule of thumb of "design effect smaller than 2" as the justification of not accounting for the multilevel or clustered structure in their data. The rule, however, has not yet been systematically studied in previous research. In the present study, we generated data from three different models…

Describes a sampling strategy which involves linked pairs of persons used to obtain bereaved respondents for a study examining loss and coping responses following a recent natural disaster. The sampling procedure appeared not to produce an obvious bias and was very beneficial in meeting the research objectives. (Author/NRB)

This chapter provides an overview of sampling methods that are appropriate for conducting online surveys. The authors review some of the basic concepts relevant to online survey sampling, present some probability and nonprobability techniques for selecting a sample, and briefly discuss sample size determination and nonresponse bias. Although some…

Possible bias due to sampling problems or low response rates has been a troubling "nuisance" variable in empirical research since seminal and classical studies were done on these problems at the beginning of this century. Recent research suggests that: (1) earlier views of the alleged bias problem were misleading; (2) under a variety of fairly…

Methods of using simplified time series analysis (STSA) in evaluating clinical programs are discussed. STSA assists in addressing problems of attrition/augmentation of subjects in programs with changing populations. Combining individually calculated "C" statistics in a simple aggregate analysis of restraint usage by nursing home staff…

Approaches to the practical problems of finding former offenders and making them want to participate in a study are described for a study of desistance from crime. In the example, a success rate of 78% was achieved for a sample of 50 former offenders. Attention to individual circumstances is critical. (SLD)

Tracking and two statistical methods (probability weighting and sample selection modeling) were studied as ways to minimize bias attributable to sample attrition in school-based studies. Data on student smoking from 30 middle schools illustrate that sample weighting yields the best results, with estimates superior to sample selection and much less…

Some students are excluded from the National Education Longitudinal Study of 1988 (NELS:88) because of an inability, whether due to physical, mental, or linguistic barriers, to participate in studies requiring questionnaire or cognitive test completion. The implications of this exclusion for sample representativeness, national estimation, and…

The extent to which students with disabilities are represented in national data collection programs was investigated by reviewing nine such programs that are receiving attention in current educational reform initiatives. Results suggest that 40-50% of students with disabilities are excluded from these prominent data collection programs. (SLD)