The use of the intraclass correlation in determining reliability is discussed and shown to be both appropriate and simple to use in the case of an observational measure, provided that observations are made on at least two occasions. The interpretation of such coefficients is explained in terms of generalizability theory, and real data are used to…

The Ohio Vocational Interest Survey (OVIS) has been used as an interest measurement tool measuring student interests toward jobs dealing with data, people and things. The purpose for the present paper was to determine the consistency of the OVIS for measuring these three dimensions through factor analysis. The research findings demonstrated which…

There are few existing or widely known measures of agreement applicable when data is nominal or categorical. Most such coefficients are applicable only when judges classify objects or subjects into a single category. A wider range of applications, including those where judges (1) place probabilities on subjects belonging to mutually exclusive and…

This paper brings together a variety of item-analysis techniques into a coherent system. The system is based on classical test theory, the theorems that can be derived from the equation, X = T + E. The system extends from techniques for analyzing parts of an item separately to techniques for relating items to total test score, to sub-scores, to…

Total circular triad scores (TCT) derived from the pair-comparison Minnesota Importance Questionnaire (MIQ) were used to study the relationship between inconsistency, and both internal consistency reliability and stability. Stability estimates (and Hoyt Coefficients) were computed for each of nine groups (retest internals from immediate retest to…

Weighting test scores by appropriateness of confidence, has almost without exception raised the reliability of test scores. Greater gains appear to occur for the less reliable tests, but that is at least partly because the more reliable a test is to begin with, the more difficult it is to improve it. If confidence-testing allows us to weight…

The evaluation of a testing program is necessary before or during a sound total project evaluation. Ideally, the testing program study should be concurrent with, and equal in magnitude to, the total project evaluation. Step one in an evaluation is to define the testing program's objectives in operational terms. Step two is a thorough description…

The variable of permissiveness developed and measured by Ira Reiss in the form of a Guttman scale of premarital sexual permissiveness was subjected to critical analysis. Both conceptual analysis and testing of questions regarding methodology and interpretation of the scale were used. Several questions were raised about the Reiss scale including:…

A filmstrip with associated audio track has been developed to cover the major planning steps in the development of a measurement instrument such as a test or questionnaire. The filmstrip addresses the following six questions: Why am I testing? What should I test? Whom am I testing? What kinds of questions should I use? How long should my test be?…

The present study attempted to modify the latencies and errors of adult women on the Matching Familiar Figures test (MFF) by systematically altering task instructions. The results indicated that latencies of impulsive subjects could be altered with "reflective" instructions, while the latencies of reflective subjects were resistent to…

For two tests measuring the same trait, the program, BIV20, equates the scores using the two True score distributions estimated by the univariate method 20 program (see Wingersky, Lees, Lennon, and Lord, 1969) and, with these equated true scores and their distributions, estimates the bivariate distribution scores and the relative efficiency of the…

The Recruit Performance Test (RPT) was developed to meet a widely recognized need for an accurate measurement of achievement in learning the military/psychomotor skills in recruit training. It was hoped that the test would be of special value in assessing the achievement of Category IV personnel, who are thought to be penalized by the present…

The item (difficulty and discrimination) and test (reliability and validity) statistics in classical test theory are highly dependent upon the calibration sample of individuals used. The estimates of item and test parameters in classical test theory is valid within a range of interest along the characteristic measured. Generally, this range of…

A major problem in the research concerning distributional and other properties of reliability coefficients has been the non-existence or inaccessibility of adequate test data for use in empirical verification of hypothetical conclusions. The purpose of this paper is to develop a technique for the simulation of test item scores through the use of…

The Diagnostic Reading Tests were designed to assess the reading skills of college students enrolled in reading centers. To assess the reliability of the Diagnostic Reading Tests, Survey Section, Form E (DRTE), a study was conducted with university freshmen as subjects. The DRTE was administered to 31 students in an Educational Opportunity Program…