There are concerns that current remote learning efforts in response to COVID19 may not be measuring up to the quality of classroom-based instruction. This study investigated two high school teachers' perceptions of the issues surrounding teaching mathematics remotely and factors that contributed to their use of technology while teaching online. The results show that both teachers found teaching mathematics online more difficult compared to classroom-based instruction. The main concerns expressed by these teachers focused on challenges related to receiving feedback from students and limited student interaction. This made it difficult to assess students' understanding during lessons. They also expressed concerns about the difficulty of implementing aspects of a task-based curriculum that relies heavily on classroom discourse and interaction; and the challenge of finding quality and usable resources for remote instruction. This study also identified several factors that influenced teachers' use of technology during COVID-19 remote learning. Implications for teacher professional development, online instruction, and policy are discussed.