An alternative derivation was given of Gaylord's formulas showing the relationships among the average item intercorrelation, the average item-test correlation, and test reliability. Certain parallels were also noted in analysis of variance and principal component analysis. (Author)

A modification of Hoyt's analysis of variance model for test analysis was proposed by Lu. A difficulty that may be encountered in using Lu's modification is examined, and a solution is proposed. (JKS)

Hoyt's analysis of variance procedure for estimating reliability assumes that the residual mean square estimates error variability. If, however, an individual's true score varies across items, it is argued that residual mean square estimates two components--error and interaction--and hence Winer's modification of Hoyt's formula, understimates the…

ANALYSIS, a FORTRAN computer program developed as an aid to investigation of and instruction in the psychometric properties of test items, is described. The program utilizes both correlational and analysis of variance approaches to classical item analysis. (Author/JKS)

An analysis of variance approach to estimating reliability is examined. This approach uses an internal-consistency index for estimating the reliability of survey instruments containing ranked items, and is recommended when the relative ranking of item means is of interest to survey researchers. The computation of this index is demonstrated using…

The validity of five methods of estimating the reliability of criterion-referenced tests was evaluated across nine conditions of variability among item means. The results were analyzed by analysis of variance, the Newman-Keuls test, and a nonparametric procedure. There was a tendency for all of the methods to be conservative. (Author/JKS)

This paper brings together a variety of item-analysis techniques into a coherent system. The system is based on classical test theory, the theorems that can be derived from the equation, X = T + E. The system extends from techniques for analyzing parts of an item separately to techniques for relating items to total test score, to sub-scores, to…

Systematically investigates the effects of test context on verbal analogy item difficulty, in terms of both simple percentage correct and easiness estimates from a parameter-invariant model (Rasch, 1960). (RC)

It has been shown that Guttman weighting of test options results in marked increases in the internal consistency of a test. However, the effect of this type of weighting on the structure of the test is not known. Hence, the purpose of this study is to compare the factor structure of Guttman-weighted and rights-only-weighted tests and to relate the…

Despite the popularity of both multiple-choice (MC) and true-false (TF) items, most investigations comparing the two formats have done so to determine the optimum number of choices to be given to students within a given time period. The purpose of this investigation was to compare the reliabilities and the validities of both formats when the items…

When classes are the units of analyses, estimates of the reliability of class means are needed. Using classical test theory it is difficult to treat this problem adequately. Generalizability theory, however, provides a natural framework for dealing with the problem. Each of four possible formulas for the generalizability of class means is derived…

The agreement between the Educational Testing Service (ETS) and the Golden Rule Insurance Company of Illinois is interpreted as setting the general principles on which items must be selected to be included in a licensure test. These principles put a limit to the difficulty level of any item, and they also limit the size of the difference in…

The comparative effects of various sequences of test items were examined for over 900 graduate students enrolled in an educational research course at The University of Alabama, Tuscaloosa. experiment, which was conducted a total of four times using four separate tests, presented three different arrangements of 50 multiple-choice items: (1)…

Eighty Singapore teachers voluntarily participated in the evaluation of three new scales: (1) the Responsibility Scale, which measures the teacher's sense of responsibility, willingness to take on additional new duties, and perception of being assigned duties; (2) the Change Scale which measures the teacher's acceptance of changes in life as…

Undergraduates in five classes were asked to rate their instructors on four general items, and on twenty-four behavior-specific items describing instructor involvement. Three reliability coefficients differing in error score assumptions were calculated for different numbers of students and items, to permit comparison of the reliabilities of both…