This study investigates the impact of course delivery methods on perceived learning by analysing the shift from face-to-face to online courses among Japanese university students, using the COVID-19 pandemic as a natural experiment. Data were collected from approximately 750 students, and detailed analyses were conducted on a subset of 426 students' data. The study controls for variations in psychological states, academic grades, and socio-economic statuses before and after the pandemic through panel data analysis. This approach minimises pandemic-related confounds, focusing on effects of online education modalities. Using random- and fixed-effects models, findings indicate asynchronous online courses significantly reduced students' perceived learning, unlike synchronous and mixed-mode courses. This impact varied by grade level, with second year students showing the most substantial decrease. These insights highlight the need for adaptive educational policies to respond to rapid transitions to online learning, aiming to preserve quality under challenging circumstances.