This study conducts the validity of the pen-and-paper and smart-device-based tests on optician's examination. The developed questions for each media were based on the national optician's simulation test. The subjects of this study were 60 students enrolled in E University. The data analysis was performed to verify the equivalence of the two evaluation methods, specifically, through split-plot factorial design of the evaluation method as a partition variable. As a result of the statistical significance test for the difference in achievement for each type of test information medium, indicating that there was no difference in achievement according to the type of test information medium at the significance level of 0.05. Although the validity of the smart device-based test and the paper-and-pencil test was verified through this study. To develop and set multimedia items in the optician national licensing examination, it is necessary to establish guidelines for how to develop the items.