Randomized experiments are commonly used to evaluate if particular interventions improve student achievement. While these experiments can establish that a treatment actually "causes" changes, typically the participants are not randomly selected from a well-defined population and therefore the results do not readily generalize. Three…

Research in fields other than education has found that studies with small sample sizes tend to have larger effect sizes than those with large samples. This article examines the relationship between sample size and effect size in education. It analyzes data from 185 studies of elementary and secondary mathematics programs that met the standards of…

Explains the characteristics and importance of random assignment in experimental research on education. Considers factors supporting causal inferences, the nature of "assignment units," differences between random sampling and random assignment, the characteristics of randomization tests, forms of random assignment, and effects of experimental…

Describes the nature and consequences of liberally or improperly applying the traditional reading research methodology and provides an argument for tempering judgments about the relative contributions that experimental studies make to the professional literature in reading. (SKC)

This article feeds into debate about the feasibility and usefulness of educational experiments by discussing methodological issues arising out of a study which sought causal links between teaching and learning of one aspect of French as a foreign language. The study involved two small-scale experiments which tested a hypothesis regarding the…

The story of the national evaluation of the Follow Through program mirrors the difficulties in trying to evaluate a complex social program that changed its community service orientation to a strategy of planned variation of early childhood programs. (Author/MLF)

THE MANY ACTIVITIES RESULTING FROM THE RECENT EMPHASIS UPON IMPROVING EDUCATION SHOULD CONCERN THE MAJOR PROBLEMS, RESEARCH TOPICS, AND PRIORITIES IDENTIFIED THROUGH A SUGGESTED SYSTEM OF VOCATIONAL EDUCATION BOOKKEEPING IN WHICH RESULTS ARE REVIEWED BY APPROPRIATE STAFF AND RESOURCE PERSONNEL FROM BUSINESS, INDUSTRY, COLLEGES, AND UNIVERSITIES.…

Advantages and disadvantages of nonexperimental paradigms in mathematics education are discussed. Suggestions are made for action toward establishing such designs as quality research. (MP)

The problem of spurious significance in multivariate exploratory research is discussed. When a very large number of statistical tests are performed, many tests will be significant on the basis of chance alone. To counter this problem, the use of two sign tests to analyze sets of results has been suggested; the chance expectation [CE test] assesses…

Abt Associates, the firm responsible for the primary analysis of the Follow Through experiment, summarize the findings. Their clearest finding indicates that local circumstances, attitudes, and activities unmeasured by evaluation have much more to do with the effectiveness of Follow Through models, without exception, than do their theoretical…

Exploratory research with large numbers of variables and even larger numbers of relationships is likely to result in frequent Type II errors; that is falsely accepting an incorrect null hypotheses. There are three ways of reducing Type II errors: (1) choosing a lower significance level, which then increases the probability of Type I error; (2)…

With an emphasis on the problems of control of extraneous variables and threats to internal and external validity, the arrangement or design of experiments is discussed. The purpose of experimentation in an educational institution, and the principles governing true experimentation (randomization, replication, and control) are presented, as are…

Studies carried out in connection with the activities of the American Educational Research Association (AERA) Committee on Access to do Research in the Public Schools.

Documenting treatment integrity is an important issue in research and practice in any discipline concerned with prevention and intervention. However, consensus concerning the dimensions of treatment integrity and how they should be measured has yet to emerge. Advances from three areas in which significant treatment integrity work has taken…

A number of potential threats to internal validity in true experiments are discussed. Programmatic research efforts proceeding from established correlational findings to experiments employing random assignment are described and used as referents in discussions of validity considerations. The collection of observational data is recommended for use…