Why would anyone think of teaching and learning mathematics directly from primary historical sources? We aim to answer this question while sharing our own experiences, and those of our students across several decades. We will first describe the evolution of our motivation for teaching with primary sources, and our current view of the advantages and challenges of a pedagogy based on teaching with primary sources. We then present three lower-division case studies based on our classroom experience of teaching discrete mathematics courses with student projects based on primary sources, and comment on how these could be adapted for use with other lower-division audiences.