Test item selection techniques based on traditional item analysis methods were compared to techniques based on item response theory. The consistency of mastery classifications in criterion referenced reading tests was examined. Pretest and posttest data were available for 945 first and second grade students and for 1796 fourth to sixth grade students who were given the PRI Reading Systems Instructional Objectives Inventory. Three traditional item analysis procedures were used: the Cox-Vargas index; the point biserial correlation applied to combined pretest-posttest data; and the phi coefficient. Two approaches derived from item response theory were also used: item information at the cut-off score as estimated by the one parameter and the three parameter models. The results indicated that the two item response models produced classification consistencies, indicated by coefficient kappa, that were superior to the three traditional procedures. Furthermore, the three parameter model appeared to be superior to the one parameter model. (GDC)