Problems in constructing criterion referenced tests related to inadequate technical criteria, and guidelines for their solution are discussed. The need to select a well-defined domain of behaviors is emphasized as the key to test construction and to understanding students' test performance. Considerations must also be given to the most effective test magnitude; that is, how much each test should cover and how many tests should be created to measure achievement in each domain. The dichotomy between the purpose of criterion referenced testing and the use of comparative data to assess student performance must also be resolved. Four guidelines to help overcome some of these limitations are: (1) the magnitude of behavior assessed should be as broad as possible without sacrificing the test's descriptive clarity; (2) test specifications should be detailed and unambiguous if there are relatively few but important behaviors being measured, even if this results in a large set of specifications; (3) estimates of required instructional time should be used to help establish a homogeneous grouping of behaviors to be measured; and (4) stimulus and response attributes of test items should be examined and their essential dimensions specified by very detailed delimiting statements. (MH)