This study explores the integration of culturally responsive mathematical modeling (CRMM) in primary grades to promote racial justice and critical consciousness. The research focuses on how CRMM can support problem-based learning (PBL) by engaging students in analyzing real-world issues through mathematics. CRMM leverages students' cultural knowledge, identities, and experiences to critically examine and address social justice issues, such as racial representation in classroom libraries. The study utilizes a case study approach to investigate the supports and challenges faced by two primary grade teachers in designing and implementing a CRMM task, the Library Diversity Project. This project aimed to evaluate and improve the fairness of classroom library collections by analyzing the representation of different racial and cultural groups in children's books. The research highlights the importance of collegial support, critical reflection, and the use of specific pedagogical tools, such as decision matrices and graphical representations, in facilitating CRMM. Key findings indicate that CRMM tasks can cultivate critical awareness and empathy among students, enabling them to take action to address inequities. The study also unveils the complexity of implementing CRMM, particularly in navigating political, contextual, and mathematical challenges. Teachers' critical reflexivity and collaboration were essential in addressing these challenges and fostering a learning environment that allowed students to use mathematics to analyze an issue and take action. The research contributes to the literature on equity-focused mathematics education by providing empirical evidence of the potential of CRMM to promote social justice in the primary grades. It emphasizes the need for ongoing professional development and support for teachers to effectively integrate CRMM into their practice. The study concludes with recommendations for future research and practice in culturally responsive mathematics teaching.