This paper describes a study that used a novel method to investigate conceptual difficulties with mathematical induction among two groups of undergraduate students: students who had received university-level instruction in formal mathematical induction, and students who had not been exposed to formal mathematical induction at the university level. The method used a 'visual proof by induction' -- a simple image that is designed to demonstrate a theorem that would be formally proven using mathematical induction -- to investigate students' conceptualizations of mathematical induction. We report two major findings: first, the majority of students who were familiar with formal mathematical induction had difficulty using the image to justify the theorem, suggesting that their knowledge of the proof method was intimately linked to the algebraic method and thus largely procedural in nature. Second, students who had not studied formal mathematical induction generally used the image as the basis of a standard inductive generalization and did not recognize that the image could be used to establish the necessity of the theorem. Surprisingly, these students often expressed conceptualizations of natural number that were inconsistent with the formal characterization that forms the basis of formal mathematical induction. Implications for education are discussed.