Purpose: This study sought to assess the introduction of a web-based innovation in medical education that complements traditional problem-based learning curricula. Utilizing the case method as its fundamental educational approach, the Interactive Case-based Online Network (ICON) allows students to interact with each other, faculty and a virtual patient in difficult neurological cases. Given the paucity of available metrics to benchmark online systems, we complement user perceptions with data on system utilization. Methods: We describe a case study of distinct, small group tutorials over 2 years as part of the Human Nervous System and Behavior (HNSB) course at the Harvard Medical School. Participating students and faculty were interviewed following completion of the course and their utilization of the system was recorded and examined. Results: Students each spent 3.2 plus or minus 1.3 h (mean plus or minus SD) through 8.6 plus or minus 2.8 accessions per week using ICON outside of required tutorial time. Faculty each spent 4.8 plus or minus 3.4 h through 16.6 plus or minus 8.9 accessions per week on ICON. Students identified real-time engagement, stronger relationships with faculty, increased accountability to the tutorial group and self-selected pace as the most beneficial characteristics of the ICON-based tutorial in comparison to traditional problem based learning (PBL) tutorials. Faculty identified enhanced collaboration with students and more realistic student experiences as the most beneficial characteristics. Both students and faculty reported that limitations of ICON included increased time investment for faculty and increased reliance on good faculty mentorship. Conclusion: This is the first study of the ICON learning system in undergraduate medical education, a platform designed to facilitate collaboration outside of the classroom. Data on user perceptions and system utilization suggest that both faculty and students chose to adopt this online learning system as a means for collaboration. The study also outlines future avenues for research in assessing novel online technologies.