This paper involves an application of generalizability theory in assessing the dependability of a foreign language placement exam. The French Cloze test was administered to students within five levels of French classes and the results were scored by four different raters. Three specific generalizability coefficients are discussed along with…

The problem that this research addressed was one of decision making. Given three sets of criterion-referenced tests which were designed to be parallel in content, would a traditional reliability coefficient produce different decisions about the reliability of those tests than would kappa? The procedure used collected statewide results on 136 test…

The use of the intraclass correlation in determining reliability is discussed and shown to be both appropriate and simple to use in the case of an observational measure, provided that observations are made on at least two occasions. The interpretation of such coefficients is explained in terms of generalizability theory, and real data are used to…

There are few existing or widely known measures of agreement applicable when data is nominal or categorical. Most such coefficients are applicable only when judges classify objects or subjects into a single category. A wider range of applications, including those where judges (1) place probabilities on subjects belonging to mutually exclusive and…

This paper brings together a variety of item-analysis techniques into a coherent system. The system is based on classical test theory, the theorems that can be derived from the equation, X = T + E. The system extends from techniques for analyzing parts of an item separately to techniques for relating items to total test score, to sub-scores, to…

Total circular triad scores (TCT) derived from the pair-comparison Minnesota Importance Questionnaire (MIQ) were used to study the relationship between inconsistency, and both internal consistency reliability and stability. Stability estimates (and Hoyt Coefficients) were computed for each of nine groups (retest internals from immediate retest to…

The present study attempted to modify the latencies and errors of adult women on the Matching Familiar Figures test (MFF) by systematically altering task instructions. The results indicated that latencies of impulsive subjects could be altered with "reflective" instructions, while the latencies of reflective subjects were resistent to…

For two tests measuring the same trait, the program, BIV20, equates the scores using the two True score distributions estimated by the univariate method 20 program (see Wingersky, Lees, Lennon, and Lord, 1969) and, with these equated true scores and their distributions, estimates the bivariate distribution scores and the relative efficiency of the…

A major problem in the research concerning distributional and other properties of reliability coefficients has been the non-existence or inaccessibility of adequate test data for use in empirical verification of hypothetical conclusions. The purpose of this paper is to develop a technique for the simulation of test item scores through the use of…

The stepped-up reliability coefficient does not have the same standard error as an ordinary correlation coefficient. Fisher's Z -transformation should not be applied to it. Appropriate procedures are suggested. (Author)

An investigation is conducted which presents extensive Monte Carlo results which indicate the conditions under which a procedure using the F distribution can be used to study the robustness of the confidence interval procedures for small samples. A review of the literature is presented. Procedure uses a binary data matrix. Results indicate that…

This study in parametric test theory deals with the statistics of reliability estimation when scores on two parts of a test follow a binormal distribution with equal (case 1) or unequal (case 2) expectations. In each case biased maximum-likelihood estimators of reliability are obtained and converted into unbiased estimators. Sampling distributions…

Results of a comparative study of F and Q tests, in a randomized block design with one replication per cell, are presented. In addition to these two procedures, a multivariate test was also considered. The model and test statistics, data generation and parameter selection, results, summary and conclusions are presented. Ten tables contain the…

This study compared a domain referenced approach with a traditional psychometric approach in the construction of a test. Results of the December, 1975 Quarterly Profile Exam (QPE) administered to 400 examinees at a university were the source of data. The 400 item QPE is a five alternative multiple choice test of information a "safe"…