This paper describes the development and efficacy of an online tool for assessing the numeracy of undergraduate students. The tool was designed to be easy to administer, provide immediate feedback to students on whether they had the required level of numeracy, and to be consistent with other measures of adult numeracy. When used with students taking a mathematics or statistics course, we found a significant correlation of r = 0.45 between their numeracy score and final mark in their enrolled course. Students who had a numeracy score less than our threshold had a 30.6% probability of failing their course, whereas students who had a numeracy score of at least our threshold had a probability of failing of only 8.0%.