Survey-based formats of assessing teaching quality in higher education are widely used and will likely continue to be used by higher education institutions around the world as various global trends contributing to their widespread use further evolve. Although the use of mobile devices for course evaluation continues to grow, there remain some unresolved aspects of the classic paper and web-based modes of evaluation. In the current study, the multigroup confirmatory factor analysis approach (MGCFA), an accepted methodological approach in general mixed-method survey research, was chosen to address some of the methodological issues when comparing these two evaluation modes. By randomly assigning one of the two modes to 33 continuing training courses at a Swiss higher education institution, this study tested whether the two different modes of assessing teaching quality yield the same results. The practical implications for course evaluation practice in institutions of higher education as well as the implications and limitations of the chosen methodological approach are discussed.