Randomized experiments are commonly used to evaluate if particular interventions improve student achievement. While these experiments can establish that a treatment actually "causes" changes, typically the participants are not randomly selected from a well-defined population and therefore the results do not readily generalize. Three…

Research in fields other than education has found that studies with small sample sizes tend to have larger effect sizes than those with large samples. This article examines the relationship between sample size and effect size in education. It analyzes data from 185 studies of elementary and secondary mathematics programs that met the standards of…

Explains the characteristics and importance of random assignment in experimental research on education. Considers factors supporting causal inferences, the nature of "assignment units," differences between random sampling and random assignment, the characteristics of randomization tests, forms of random assignment, and effects of experimental…

This article feeds into debate about the feasibility and usefulness of educational experiments by discussing methodological issues arising out of a study which sought causal links between teaching and learning of one aspect of French as a foreign language. The study involved two small-scale experiments which tested a hypothesis regarding the…

The problem of spurious significance in multivariate exploratory research is discussed. When a very large number of statistical tests are performed, many tests will be significant on the basis of chance alone. To counter this problem, the use of two sign tests to analyze sets of results has been suggested; the chance expectation [CE test] assesses…

Exploratory research with large numbers of variables and even larger numbers of relationships is likely to result in frequent Type II errors; that is falsely accepting an incorrect null hypotheses. There are three ways of reducing Type II errors: (1) choosing a lower significance level, which then increases the probability of Type I error; (2)…

A number of potential threats to internal validity in true experiments are discussed. Programmatic research efforts proceeding from established correlational findings to experiments employing random assignment are described and used as referents in discussions of validity considerations. The collection of observational data is recommended for use…

The purpose of this paper is to summarize and evaluate experiments which examined the effects of teacher questions on student achievement. The studies reviewed are of two types: (a) training experiments, in which the independent variable is teacher training; and (b) skills experiments, in which the frequency and manner of use of a teaching skill…

The technique originating from the study of ecological systems for the examination of dynamic educational systems over time is utilized. The technique is based on the theory of energy exchanges in interactions between variables. Using this technique, interactions among variables can be specified and simulated over time using an analog computer.…

Discusses the need for awareness of real and potential problems in educational evaluation approaches, kinds of problems there are with true experimental designs and with quasi-experimental designs, and the criteria applied by the Joint Dissemination Review Panel to its review of educational programs. (MF)

The paper discusses a number of issues concerning the practical usefulness of cognitive psychology research, and presents a case study of pilot training methods to illustrate a model of research processes that produces outcomes which contribute to both basic and applied research goals. Research studies are described as varying in the degree to…