Several issues relating to agreement estimates for different types of data from performance evaluations are considered. New indices of agreement are presented for ordinal level items and for summative scores produced by nominal or ordinal level items. Two sets of empirical data illustrate the performance of the two formulas derived to estimate…

Given "T" joint observations on "K" variables, it is frequently useful to consider the weighted average or scaled score. L-scaling is introduced as a technique for determining the weights. The technique is so named because of its resemblance to the Leontief matrix of mathematical economics. L-scaling is compared to two…

The assumptions and consequences of applying conventional and newer statistical methods to meta-analytic data sets are reviewed. The application of the two approaches to a meta-analytic data set described by L. V. Hedges (1984) illustrates the differences. Hedges analyzed six studies of the effects of open education on student cooperation. The…

A study investigated the reading comprehension and educationally related characteristics of the United States Navy's recruit population based on data for selected variables. The variable of primary interest was the recruit's grade level score on reading comprehension; other variable employed in the study were grade level score, years of education,…

Critical Incident Technique (CIT) involves the collection of real-world examples of behavior that characterize either very effective or very ineffective performance of some activity. The principal advantage of the CIT is that it generates data based on actual behavior rather than on a particular researcher's subjectivity. The CIT has much to offer…

This book presents a compilation of 28 issues of the newsletter, "How To Evaluate Education Programs" from the first one published in September 1977 through the issue of December 1979 on the topic of evaluating educational programs. The subject is covered in the following chapters: (1) How to Choose a Test; (2) How to Rate and Compare…

Meta-analysis was used to conduct an integrative review of the research on modifying attitudes toward persons with disabilities. Prior reviews, using small samples and lacking systematic data collection and analysis, were unable to draw firm conclusions about the effectiveness of attitude modification techniques. In the hope of correcting these…

Data and information needs of college administrators are discussed, along with the uses of information to support decisionmaking. Difficulties in applying information technology to the administration of higher education are briefly reviewed. Data and information are discussed both in definitional and functional terms. Data are defined as either…

The focus of this study was on the visual analysis of time series data for evaluating educational programs. Two characteristics of the data--changes in slope and variability--and two characteristics of evaluation--training in data utilization and the use of aimlines/decision rules--were manipulated. A total of 51 students and/or teachers in…

A Rasch measurement model can be constructed to meet the requirements of rank ordered data. If multiple rankings of the same objects are available, then the parameters of the objects can be estimated, along with their standard errors and also with statistics summarizing the fit of the data to the measurement model. This paper summarizes the…

This document provides information on the Data on Vocational Education (DOVE) plan, which has provided the National Center for Education Statistics (NCES) with a framework on which a viable data collection and dissemination program is being constructed for vocational education. A section on the status of DOVE discusses the attainment of the…

A study was conducted to investigate the reading comprehension and educationally related characteristics and relationships that describe the Navy's recruit population. The variable of primary interest was the recruit's reading comprehension grade level score; other variables were years of education, aptitude score, high school graduation status,…

Various methods have been suggested for the analysis of data collected in research settings where random assignment of subjects to groups has not occurred. For the purposes of this paper the set of allowable nonrandomized designs is made up of those research designs where data are collected for one or more groups of subjects at two or more time…

Designed to assist persons who need to compare and use information obtained under various classification systems, this volume links the Department of Education vocational program categories in the new Classification of Instructional Programs (CIP) to the occupations in the Dictionary of Occupational Titles (DOT), the Standard Occupational…

The report addresses the question of the structure of cognitive abilities, i.e., how many different abilities should be identified and the relations between abilities. Models suggested by Spearman, Thurstone, Guilford, Vernon, Cattell-Horn and others are reviewed. By combining features of the Vernon and Cattell-Horn models it is possible to…