Artificial intelligence (AI) shows increasing potential to improve mathematics instruction, yet integrative quantitative evidence currently is lacking on its overall effectiveness and factors influencing success. This systematic review and meta-analysis investigate the effectiveness of AI on improving mathematics performance in K-12 classrooms compared to traditional classroom instruction. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we searched five databases from 2000 through December 2023, synthesizing findings from 21 relevant studies (40 samples) which met screening criteria. Results indicate a small overall effect size of 0.343 favoring AI under a random-effects model, showing a generally positive impact. Only one variable, AI type, was identified as having moderate effects, with AI demonstrating a greater impact when served as an intelligent tutoring system and adaptive learning system. Our findings establish an initial knowledge base for implementation and future research on the effective integration of AI into K-12 mathematics classrooms. This study also focuses on the appropriateness across age, mathematical contents, and AI design factors is aimed at further advancing the judicious adoption and success of classroom AI integration.