Background: Poor mathematics performance in South Africa is well known. The COVID-19 pandemic was expected to exacerbate the situation. Aim: To investigate Grade 7 learners' mathematical knowledge at the end of primary school and to compare mathematical performance of Grade 7 and 8 learners in the context of the pandemic. Setting: Data were collected in term four of 2020 at 11 primary schools and five secondary schools. All schools drew learners from poor communities in Gauteng. Methods: A multiple-choice test covering mathematical content from Grades 4-7 was designed and piloted. Learner performance was measured through number of correct responses. Qualitative error analyses were conducted on learners' choices of distractors. Results: The difference in performance of the two grade groups was not statistically significant. There were similar response patterns in learners' choices of distractors with strong evidence of cue-based reasoning and evidence of additive reasoning in items requiring multiplicative reasoning. Conclusion: Grade 8 learners made very small gains, likely due to reduced learning time. Learner errors show many similarities with the international literature and show that Grade 7 learners are not yet ready for algebra. Contribution: The findings provide starting points for addressing the most common errors and highlight the need for: greater attention to whole and rational number concepts in Grade 8; strategies for teacher support in teaching primary maths content; and innovative teaching strategies to fast-track learning of this content.