This study empirically examined the proposition that different item selection procedures contribute to the development of achievement tests that optimally serve different purposes. A test of 110 addition and subtraction problems was given to a sample of 371 first-grade students. Three subsets of 30 items were selected from this 110-item test. The first included those items with the highest classical item discrimination indexes, the second of those best describing the continuum of Rasch item difficulties of the 110-item test, and the third of randomly selected items. The study examined which set of items best reflected student performance on the set of 110 items if the purpose of such testing was the differentiation of students, location of students on an achievement continuum, or generalizing student performance to the domain of 110 items. Primary analyses of the data indicated that the subtests consisting of classically discriminating items and those describing the continuum of Rasch difficulties yielded statistically different results from all other tests for differentiating students or generalizing performance to the domain of items. Secondary analyses indicated that these differences were attributed to the performance of the lower quartiles of students. (Sixteen tables are appended.) (Author)