The resources and time constraints of assessing large classes are always weighed up against the validity, reliability, and learning outcomes of the assessment tasks. With the digital revolution in the 21st Century, educators can benefit from computer technology to carry out a large-scale assessment in higher education more efficiently. In this article, an in-depth case study of a nursing school that has integrated online assessment initiatives into their nursing program. To assess a large class of first-year nursing students, a series of non-proctored multiple-choice online quizzes are administered using a learning management system. Validity and reliability are commonly used to measure the quality of an assessment. The aim of the present article to analyze these non-proctored multiple-choice online assessments in the context of content validity and reliability. We use this case study to examine online assessment in nursing education, exploring the benefits and challenges. We conclude that instructors have to determine how to use the full potential of online assessment as well as ensure validity and reliability.