The instructional validity of computer administered tests was studied with a focus on whether differences in test scores and item behavior are a function of instructional mode (computer versus non-computer). In the first of 3 studies, performance test scores for approximately 400 high school students in 1990-91 for tasks accomplished with the computer were correlated with objective paper-and-pencil test scores for comprehension of the same tasks. In the second study, a comparison was made between test scores of 77 high school students in 1991-92 taking an objective test via computer and scores on the same test in paper-and-pencil format. In the third study, the item characteristics of tests given in a computer format were compared to the item characteristics of the same tests presented in a paper-and-pencil format. Data from the first two studies were used. Findings from the three studies indicate that performance tests for computing are not equivalent to objective paper-and-pencil tests even when the content is the same. Test scores and item statistics for tests in a paper-and-pencil format and computer format do not differ for content taught in a combined computer and lecture mode. Whether students not taught by computer are hampered by computer-based tests is a logical extension of this research. Seven tables present study findings. (SLD)