This study examined the academic performance and preference of students with disabilities for two types of test administration conditions, computer-based testing (CBT) and pencil-and-paper testing (PPT). Data from a large-scale assessment program were used to examine differences between CBT and PPT academic performance for third to eleventh grade students with a read-aloud accommodation in reading, mathematics, and science. Since random assignment was not possible, propensity score analyses were used to establish equivalent groups and to test for differences in performance. Students in the PPT condition with an adult reader had higher mean scores in almost all academic content areas than those with the CBT read-aloud condition, with effect sizes ranging from extremely small (d = 0.02) to moderate (d = 0.69). Differential item functioning (DIF) analyses suggested that most items had negligible DIF and did not favor either the CBT or the PPT conditions. Students and staff reported that students preferred the CBT to the PPT, and students believed they performed better using the computer. Additional research that controls extraneous factors such as instructional time and familiarity with testing environment is recommended to better evaluate the relationship between testing modes and academic performance. (Contains 6 tables.)