The present study was designed to examine whether or not traditional procedures concerning item selection and reliability are both applicable and appropriate for criterion-referenced (CR) tests. It was also designed to examine traditional procedures and those designed especially for CR testing in relation to test variance and item homogeneity. Specifically, the following questions were formulated: (1) How are traditional and criterion-referenced item selection techniques interrelated? (2) How are traditional and criterion-referenced reliability indices interrelated? (3) To what degree does variability among subjects and homogeneity of items affect traditional and criterion-referenced item selection procedures and reliability indices? The results of the study indicate that the construction of CR tests can be greatly facilitated by item analysis procedures such as phi, and the application of traditional reliability estimates such as KR-20. The procedure of trying out test items on a group of examinees consisting of master and nonmasters is recommended. (Author)