The use of large language models like ChatGPT is widely discussed for educational purposes. Using this technology requires teachers to have appropriate competences that incorporate knowledge of how to make use of this technology. In this study, we investigate pre-service teachers' knowledge through the lens of the KTMT model ("Knowledge for Teaching Mathematics with Technology" model), a domain-specific variant of the TPACK-model. One component is represented in mathematical fidelity as knowledge of the mathematical accuracy of the technology, which in case of large language models is of special interest, as it may produce erroneous but plausible-sounding information. Furthermore, prompting techniques are of interest as technological knowledge, which influence mathematical fidelity. For this study, eleven pre-service teachers were asked to solve four different mathematical tasks with the help of ChatGPT. The chatlogs and information provided in an interview after working on the tasks are analyzed using qualitative content analysis. Results show that both correct and incorrect answers were produced for all tasks. The rate of pre-service teachers providing an incorrect answer is high when having been presented with an incorrect answer generated by the large language model. Despite having access to ChatGPT as a tool, many of the participants were not able to provide correct answers to all tasks. Furthermore, the mathematical fidelity was often over- and, in some cases, underrated. The mathematical knowledge seems to have changed while working with ChatGPT. Based on the applied prompting techniques, the pre-service teachers showed a deficiency in technological knowledge.