Over the past decade, propensity score-based methods have made an important contribution to the improvement of generalizations from educational studies. However, an important limitation is that many studies are based on small sample sizes in which the study comprises 3-5% of the inference population. In response, one suggestion that has been made…

Not knowing the population size is a common problem in data-limited contexts. Drawing on work in Sierra Leone, this short take outlines a four-step solution to this problem: (1) estimate the population size using expert interviews; (2) verify estimates using interviews with participants sampled; (3) triangulate using secondary data; and (4)…

In educational contexts, students often self-select into specific interventions (e.g., courses, majors, extracurricular programming). When students self-select into an intervention, systematic group differences may impact the validity of inferences made regarding the effect of the intervention. Propensity score methods are commonly used to reduce…

Few empirical studies exist to guide researchers in determining the number of focus groups necessary for a research study. The analyses described here provide foundational evidence to help researchers in this regard. We conducted a thematic analysis of 40 focus groups on health-seeking behaviors of African American men in Durham, North Carolina.…

Despite recent emphasis on the use of randomized control trials (RCTs) for evaluating education interventions, in most areas of education research, observational methods remain the dominant approach for assessing program effects. Over the last three decades, the within-study comparison (WSC) design has emerged as a method for evaluating the…

Formal inference, which makes theoretical assumptions about distributions and applies hypothesis testing procedures with null and alternative hypotheses, is notoriously difficult for tertiary students to master. The debate about whether this content should appear in Years 11 and 12 of the "Australian Curriculum: Mathematics" has gone on…

In 4 studies, the authors examined the hypothesis that the structure of the informational environment makes small samples more informative than large ones for drawing inferences about population correlations. The specific purpose of the studies was to test predictions arising from the signal detection simulations of R. B. Anderson, M. E. Doherty,…

This research examined the issue of nonresponse bias and how it was reported in nonexperimental quantitative research published in the "Delta Pi Epsilon Journal" between 1995 and 2004. Through content analysis, 85 articles consisting of 91 separate samples were examined. In 72.5% of the cases, possible nonresponse bias was not examined in the…

To ensure accurate interpretation of research findings, researchers should report details about their research design, data collection method, and response rates when presenting findings from survey research. A review of 100 peer-reviewed articles reporting the results of survey research on K-12 schools with principals as the designated…

This chapter provides an overview of sampling methods that are appropriate for conducting online surveys. The authors review some of the basic concepts relevant to online survey sampling, present some probability and nonprobability techniques for selecting a sample, and briefly discuss sample size determination and nonresponse bias. Although some…

The National Household Education Survey (NHES) is a data collection system of the National Center for Education Statistics. The NHES is a telephone survey of the noninstitutionalized civilian population using households selected using random digit dialing methods. Approximately 60,000 households are screened for each administration, and people who…

Induced probabilities have been largely ignored by educational researchers. Simply stated, if a new or random variable is defined in terms of a first random variable, then induced probability is the probability or density of the new random variable that can be found by summation or integration over the appropriate domains of the original random…

The National Education Longitudinal Studies Program (NELS) is a long-term effort that encompasses the educational experience of youth from three decades. This report is based on the National Education Longitudinal Study of 1988 (NELS:88), a component of the NELS series that began as a national probability sample of eighth grade schools and…