The desirable characteristics of criterion referenced test items and sets of items are described. A two-stage item tryout and item selection procedure are also described. The paper presents the results of using the procedure as compared with traditional item selection procedures used in selecting items for norm referenced tests. It was found that the items selected from the same item pool by the two procedures differ markedly. A rationale for these differences is presented and recommendations for appropriate uses of the two kinds of instruments are given. (Author)