Given the growing trend toward using technology to assess student learning, this investigation examined test mode comparability of student achievement scores obtained from paper-pencil and computerized assessments of statewide End-of-Course and End-of-Grade examinations in the subject areas of high school biology and eighth-grade English Language Arts and math. Propensity score matching was used to generate comparable groups of students who were assessed using paper-pencil or computer-based formats. T-tests and generalized linear models were further used to examine test mode effect. Analyses revealed a small test mode effect for all three subjects such that students using the paper-based format achieved higher scores than students using the computer-based format. The findings are germane to school districts transitioning to computerized assessments and investigating test mode comparability.