Despite evidence for the involvement of working memory in mathematics attainment, the understanding of its components relationship to individual areas of mathematics is somewhat restricted. This study aims to better understand this relationship. Two-hundred and fourteen year 3 children in the UK were administered tests of verbal and visuospatial working memory, followed by a standardised mathematics test. Confirmatory factor analyses and variance partitioning were then performed on the data to identify the unique variance accounted for by verbal and visuospatial working memory measures for each component of mathematics assessed. Results revealed contrasting patterns between components, with those typically visual components demonstrating a larger proportion of unique variance explained by visuospatial measures. This pattern reveals a level of specificity with regard to the component of working memory engaged depending on the component of mathematics being assessed. Implications for educators and further research are discussed.