The effects of test-item selection on total test reliability and validity were investigated. It was posited that in a reading comprehension test, the knowledge displayed by the examinees is of interest only as it is a valid measure of how much a student learned from reading or comprehending a stimulus paragraph. Selection of items solely on the basis of the items' difficulty and discrimination is not sufficient, since information gain must be considered. Using procedures developed by Marks and Noll for evaluating reading and listening comprehension tests (administration of items without stimulus paragraphs and comparison to results from a later normal administration of the test), the effect of deleting poor items on test reliability was studied. A nonrandom sample of college students (130) and a randomly selected high school sample (110) were administered 36 multiple-choice items from the comprehension subtest of the Nelson-Denny Reading Test, Form A. Later the complete comprehension subtest was given. A cloze test as a validity criterion was developed from the comprehension subtest and given to a random sample of 150 high school students. While deletion of poor items did not increase total test reliability, the implications were that criteria beyond item difficulty and discrimination are required in selecting reading comprehension test items. Tables, references, and test items are included. (WB)