Two approaches to criterion-referenced test construction are compared. Classical test theory is based on the practice of random sampling from a well-defined domain of test items; latent trait theory suggests that the difficulty of the items should be matched to the achievement level of the student. In addition to these two methods of test construction, the independent variables of the study were test length and type of criterion-referenced test data, varying in sensitivity to instruction. The dependent variables of the study included two indices of the amount of measurement error present in a set of test scores. The results were consistent across four data sets. Tests created by selecting appropriate difficulty levels for students based on the Rasch model yielded smaller errors of measurement than tests which were created by randomly sampling items. This study also indicated that the relationship between measurement error and test length is a curvilinear function with the greatest decrease in error ocurring between 10 and 20-item tests. (BW)