This paper reports an intervention study in the teaching of fraction concepts at a village school in Hong Kong. The classes consisted of 60 Grade three students with different cultural backgrounds, including African, Chinese, Filipino, Korean, Nepalese, and Pakistani. The study represents one of the relatively few studies that investigate the effectiveness of storytelling in teaching mathematics to ethnic minority (EM) students in the East Asian context. The quantitative analysis in a pre/post-test intervention/control group design shows reduction in achievement gap between EM students in rural settings and their Chinese peers in urban districts, including concepts such as part--"whole interpretation," "fraction language," and "mathematics vocabulary." Analysis of classroom discourse based on the culturally responsive mathematics teaching (CRMT) framework reveals the multimodal features of storytelling provide greater access for EM students in acquiring language-related conceptualizations of fractions. Based on our analysis, we argue that mathematics teaching is inseparable from language teaching, which is often ignored or not planned explicitly in the lesson. Storytelling integrates language with mathematics content authentically and allows students to cope with the language demands in learning mathematics and to encounter the mathematics in real-world context.