Chatbots represent a promising technology for engaging students in math learning. Guided by Jerome Bruner's constructivism and Lev Vygotsky's Zone of Proximal Development, we designed and developed a chatbot that incorporates scaffolding strategies and social-emotional considerations, and we integrated it into ASSISTments, an online math learning platform. We conducted an experimental study to examine the influence of learning math with the chatbot compared to traditional learning with hints. This study involved 85 middle and high school students from three diverse school settings in the United States. The results revealed no significant differences in students' math learning performance and perceived helpfulness and interest between the chatbot and traditional hints conditions. However, students in the chatbot condition displayed significantly lower confidence in solving a similar problem after the intervention, likely due to the removal of the high level of support provided by the chatbot. Despite this, students' open responses indicated that a significantly higher number of students had positive attitudes towards chatbots. They appreciated the chatting feature, breaking down a problem into steps, and real-time support. The study concludes with a discussion of the findings and implications for chatbot designers and developers and presents avenues for future research and practice in chatbot-assisted learning. In support of Open Science, this study has been preregistered and both the data and the analysis code used in this study are publicly available at https://osf.io/am3p8/.