Researchers often use generalizability theory to estimate relative error variance and reliability in teaching observation measures. They also use it to plan future studies and design the best possible measurement procedures. However, designing the best possible measurement procedure comes at a cost, and researchers must stay within their budget…

A concern for generalization dominates quantitative research. For generalizability and repeatability, identification of sample size is essential. The present study investigates 90 qualitative master's theses submitted for the Primary and Secondary School Science and Mathematics Education Departments, Mathematic Education Discipline in 10…

Methodological barriers have been highlighted as a primary reason for the limited research with lesbian, gay, and bisexual (LGB) people of color. Thus, strategies for anticipating and addressing potential methodological barriers are needed. To address this need, this article discusses potential challenges associated with conducting research with…

Views intercoder reliability as a sampling problem for content analyses. Develops a formula for generating sample sizes needed to have valid reliability estimates. Suggests steps for reporting reliability. (TB)

A computer simulation was undertaken to determine the effects of using Huynh's single-administration estimates of the decision consistency indices for agreement and coefficient kappa, under conditions that violated the beta-binomial assumption. Included in the investigation were two unimodal score distributions that fit the model and two bimodal…

Arguments have recently been put forth that standard textbook procedures for determining the sample size necessary to achieve a certain level of power in a completely randomized design are incorrect when the dependent variable is fallible because they ignore measurement error. In fact, however, there are several correct procedures, one of which is…

The purpose of this study is to help define the precise nature and limits of the tolerable range in which a researcher may be relatively confident about the statistical validity of his or her research findings, focusing specifically on the statistical validity of results when violating the assumptions associated with the one-way, fixed-effects…

The central purpose of the present study was to evaluate several statistical techniques (analysis of variance (ANOVA) on raw gains, standardized ANOVA, standard analysis of covariance (ANCOVA), and z-score ANOVA) under various quasi-experimental conditions (three levels of reliability, three levels of sample size, three levels of gain, and the…

This study by two Ph.D. students in library science reviewed a systematic sample of the research literature in library and information science published in 62 core library journals in 1983 to order to identify substantive articles and compare their findings on research methodologies and trends with those reported in a similar study performed in…

Researchers have been limited in their ability to examine multiple constructs simultaneously due to the constraints imposed by traditional statistical methods. The most notable limitations include the need for a relatively large sample size while restricting the variables to a relatively small number. The application of a newly discovered…

This paper evaluates the logic underlying various criticisms of statistical significance testing and makes specific recommendations for scientific and editorial practice that might better increase the knowledge base. Reliance on the traditional hypothesis testing model has led to a major bias against nonsignificant results and to misinterpretation…

Statistical significance is misused in educational and psychological research when it is applied as a method to establish the reliability of research results. Other techniques have been developed which can be correctly utilized to establish the generalizability of findings. Methods that do provide such estimates are known as invariance or…

The nine-volume Survey Kit is designed to help readers prepare and conduct surveys and become better users of survey results. All the books in the series contain instructional objectives, exercises and answers, examples of surveys in use, illustrations of survey questions, guidelines for action, checklists of "dos and don'ts," and…

The statistical properties of two methods of estimating gain scores for groups in quasi-experiments are compared: (1) gains in scores standardized separately for each group; and (2) analysis of covariance with estimated true pretest scores. The fan spread hypothesis is assumed for groups but not necessarily assumed for members of the groups.…

The National Household Education Survey (NHES) was conducted for the first time in 1991 as a way to collect data on the early childhood education experiences of young children and participation in adult education. Because the NHES methodology is relatively new, field tests were necessary. A large field test of approximately 15,000 households was…