In this study, we describe what factors influence the observed score correlation between an (external) anchor test and a total test. We show that the anchor to full-test observed score correlation is based on two components: the true score correlation between the anchor and total test, and the reliability of the anchor test. Findings using an…

Current methods for obtaining reliability indices for mastery tests can be laborious. This paper offers practitioners tables from which agreement and kappa coefficients can be read directly and provides criterion for acceptable values of agreement and kappa coefficients. (TJH)

It is proposed that the reliability of criterion-referenced test scores be defined in terms of the consistency of the decision-making process across repeated administrations of the test. (Author/RC)

When comparing two tests that measure the same trait, separate comparisons should be made at different levels of the trait. A simple, practical, approximate formula is given for doing this. The adequacy of the approximation is illustrated using data comparing seven nationally known sixth-grade reading tests. (Author/RC)

Some procedures for estimating internal consistency reliability may be superior mathematically to the more commonly used methods such as Coefficient Alpha. One problem is computational difficulty; the other is the possibility of overestimation due to capitalization on chance. (Author/CTM)

Huynh (1976) suggested a method of approximating the reliability coefficient of a mastery test. The present study examines the accuracy of Huynh's approximation and also describes a computationally simpler approximation which appears to be generally more accurate than the former. (Author/RL)

Different indices for the internal consistency, reproducibility, or homogeneity of a test are based upon highly similar conceptual frameworks. Illustrations are presented to demonstrate how the maximum and minimum values of KR20 are influenced by test difficulty and the shape of the distribution of test scores. (Author/CTM)

Ten item discrimination indices, across a variety of item analysis situations, were compared, based on the validities of tests constructed by using each of the indices to select 40 items from a 100-item pool. Item score data were generated by a computer program and included a simulation of guessing. (Author/CTM)

The reliabilities of various observational measures were determined, and the influence of both the number and the length of the observation periods on reliability was examined, both separately and jointly. A single simplifying assumption leads to a variant of the Spearman-Brown formula, which may have wider application. (Author/CTM)

This review of issues about correcting for attenuation concludes that the basic difficulty lies in being able to identify and equate sources of variance in estimates of validity and reliability. Recommendations are proposed for cautious use of correction for attenuation. (Author/CM)

A sample of the literature on test speededness is reviewed; methods of assessing speededness are presented and criticized; the assumptions that underlie these methods are questioned, and alternate, multiple-administration methods are suggested. The importance of the effect of time limits is discussed. (Author/CTM)

A dozen different approaches that yield 13 reliability indices for criterion-referenced tests were identified and grouped into three categories: threshold loss function, squared-error loss function, and domain score estimation. Indices were evaluated within each category. (Author/RL)