The problem that this research addressed was one of decision making. Given three sets of criterion-referenced tests which were designed to be parallel in content, would a traditional reliability coefficient produce different decisions about the reliability of those tests than would kappa? The procedure used collected statewide results on 136 test pairs. Each test in a pair consisted of five items; and each test measured a performance objective that was considered to be minimal by curriculum and measurement specialists. Three samples of test pairs were selected: one represented the range of the parallel forms definition of reliability; one the Kuder-Richardson Formula 20 (KR-20) definition; and one sample represented the range of kappa. Parallel forms, KR-20, and kappa were the three levels of the independent variable. The two dependent variables were the mean KR-20 for each item pair and the kappa for each item pair. One analysis of variance was performed to find out if a difference existed among the mean KR-20's for the three groups; a second analysis of variance was performed to find out if a difference existed among the mean kappas for the three groups. No significant difference was found among the KR-20's for the three groups. A significant difference was found among the mean kappas for the three groups. The Tukey Post Hoc method of group mean comparison revealed that both the KR-20 and the kappa group differed from the parallel forms group, but they did not differ significantly from each other. The obtained value of kappa does seem to be related to the homogeneity of the items in a test, as measured by KR-20, but it was not concluded that KR-20 could be in any way predicted from kappa, at least based on this study. (Author/MV)