College Level Examination Program (CLEP) Tests were normed on a national basis, administering the test to nationwide samples of subjects. Norms appear in the booklet, CLEP Scores: Interpretation and Use, and consist of the test score means for groups of students receiving grades of A, B, C, D, and F in the relevant course, the proportion of students receiving each grade, the sample size, and the correlation coefficient between test scores and earned course grades. The validity of the tests for the purpose of selecting creditable students from the population of test takers is assessed through an examination of the systematic differences in test means across earned grades, and the correlation coefficient between earned grades and test scores. This paper supports the assertion that the correlation coefficient is a misleading statistic for the purpose of validity assessment in this context. A decision theoretic procedure is developed which focuses on the likelihood of errors in test based decisions. The decision theoretic means of validity assessment was applied to the data of all CLEP tests discussed in the norming literature. The analysis showed a dramatically incoherent value system displayed among the tests, with wildly fluctuating error likelihoods and ratios. Further, the likelihood of each error type was found to be substantially greater than would be supposed based on an examination of the correlation coefficients between earned grades and test scores.(Author/MV)