The paper introduces the notion of collateral creativity as an unintended outcome of the appropriate use of technology, both physical and digital, in the teaching of mathematics. The notion is motivated by Dewey's conceptual framework of collateral learning which enables one gaining knowledge beyond the intent of the traditional curriculum. Creative thinking by the learners of mathematics may emerge accidentally, in a collateral way, within a classroom culture encouraging reflection on a teacher-assisted technological representation of mathematical concepts. The use of technology is described in terms of Vygotsky's concept of the instrumental act. The main argument of the paper that tokens of collateral creativity can be observed in all students resides at the confluence of various theories which, in the context of mathematics, converge to describe creativity as a slumberous skill with potential to be awakened through the age-appropriate pedagogical mediation supported by the teachers' awareness of often hidden complexity of seemingly mundane problems. The paper discusses different scenarios conducive for the emergence of collateral creativity through the study of mathematics with physical and digital tools. The ideas of the paper stem from the authors' work with schoolchildren and with teacher candidates both in the classroom and in the field.