For STEM faculty to approach teaching as a scientist, they must develop tools for data collection and analysis of student learning outcomes. Here, we report a methodology for the development of a learning outcomes assessment instrument and statistical analysis of that instrument that can be undertaken in a short amount of time by a few faculty members with little to no funding. Our team of instructors at a public land-grant university developed an instrument for our single-semester nonmajors biochemistry course. The instrument consists of eight sets of multiple true/false questions assessing learning objectives covering topics within protein structure and function, thermodynamics, and metabolism. We employed the instrument as a pre- and postcourse evaluation for several semesters. We conducted statistical analyses on overall exam scores and on individual questions. The results indicate that between the beginning and the end of the semester students achieved statistically significant increases in their cumulative scores. Finer-grained analysis revealed that students displayed little to no improvement in specific content areas and concepts. These findings point to areas in need of pedagogical interventions.