Randomized controlled trials (RCTs) admit unconfounded design-based inference--randomization largely justifies the assumptions underlying statistical effect estimates--but often have limited sample sizes. However, researchers may have access to big observational data on covariates and outcomes from RCT non-participants. For example, data from A/B…

The current study investigates the relative extent to which information utility and planning efficiency guide information-sampling strategies in a classification task. Prior research has pointed to the importance of probability gain, the degree to which sampling a feature reduces the chance of error, in contexts where participants are restricted…

Sampling tasks and sampling distributions provide a fertile realm for investigating students' conceptions of variability. A project-designed teaching episode on samples and sampling distributions was team-taught in 6 research classrooms (2 middle school and 4 high school) by the investigators and regular classroom mathematics teachers. Data…

A classroom exercise for undergraduate and beginning graduate students that takes about one class period is proposed and discussed. It is an easy, interesting exercise that demonstrates important aspects of sampling techniques (sample amount, particle size, and the representativeness of the sample in relation to the bulk material). The exercise…

This article examines theoretical and empirical issues related to the statistical power of impact estimates for experimental evaluations of education programs. The author considers designs where random assignment is conducted at the school, classroom, or student level, and employs a unified analytic framework using statistical methods from the…

Expresses the importance of scientifically based research in education reform and explains how to judge the validity of educational research. Describes control groups, randomized and matched experiments, statistical educational significance, sample size, and the difference between scientifically based and rigorously evaluated research. (Contains…