Test construction is not the strictly logical process that we might wish it to be. This is particularly true in a large on-going project such as the National Assessment of Educational Progress (NAEP). Most of the really deep questions can only be answered by the exercise of well-informed human judgment. Criterion-referenced testing is still a term in search of definition. It has been suggested that NAEL's exercises might be more properly called "objective referenced" tests. That is a reasonable title for our efforts since we are attempting to assess the degree of achievement of stated goals without reference to a predetermined level or criterion. Whatever the appropriate title may be, we share the concerns of all workers in the field with the same basic questions. But until satisfactory scientific solutions have been found; we, like the rest of education, must rely on the best human judgment available. (Author) Primary type of information provided by report: Procedures (Exercise Development).