A student-led mathematics bootcamp has been designed and implemented to help foster community building, improve confidence in mathematical skills, and provide mathematical resources for incoming physical chemistry doctoral students. The bootcamp is held immediately before the start of the first semester of graduate school and uses an active learning approach to review and practice undergraduate-level mathematics problems over 5 days in small student groups. This work includes the development and presentation of a new, publicly available mathematics curriculum for the bootcamp on select mathematics topics, including calculus, linear algebra, functions, differential equations, statistics, and coding in Python, aiming at improving students' confidence and learning experiences in graduate quantum mechanics and statistical physics courses. Surveys before and after the bootcamp showed an increase in students' confidence in problem-solving in key mathematical areas and social aspects of peer-led group learning. Qualitative and quantitative analyses demonstrate that the bootcamp reduced prior inequities in students' confidence metrics based on gender and mathematical background.