This study compared a domain referenced approach with a traditional psychometric approach in the construction of a test. Results of the December, 1975 Quarterly Profile Exam (QPE) administered to 400 examinees at a university were the source of data. The 400 item QPE is a five alternative multiple choice test of information a "safe" physician should know. Content of the exam covers the broad areas of Internal Medicine, Pediatrics, Obstetrics/Gynecology, Surgery, and Basic Science, as well as additional sub-topics. For purposes of this study, two 75 item tests were constructed by pulling from the 400 item QPE by two different strategies. The domain referenced approach was used to construct a 75 item test by a random sample of the 400 items. Selection of the 75 items with the highest point biserial item-total correlations represented the traditional psychometric approach to test construction. The exams were then rescored to obtain scores and item analysis data on the random and psychometric tests. Then, the two tests were compared with respect to distribution of p values (the proportion answering an item correctly), point biserial item-total correlations, student scores across medical school year level and reliability. The results were discussed with regard to their consistency with expectations of the domain referenced and psychometric approaches. (Author/RC)