The association between the observable physical world and the mathematical models used in theoretical physics to describe this world is examined. Such models will frequently exhibit solutions that are "unexpected," in the sense that they describe physical situations which are different from that which the physicist may initially have had in mind when the model is employed. It will then often be the case that such an unexpected solution actually represents a physically realisable situation. However, this solution may also, in some cases, describe a system that is "unphysical"--i.e. a system that, according to our physical intuition, simply cannot exist. It is argued that this latter fact poses a problem for the epistemic position of "realism" in physics: a mathematical theory that professes to give a correct description of physical reality should not give rise to such unrealistic solutions. Some concrete examples are examined, and their implications are discussed from the "relativist" epistemic perspective that is implicit in the theory of radical constructivism. It is suggested that the occurrence of these unphysical solutions also has significance for the teaching of physics. In conclusion, it is advocated that the students be exposed, from the start of their physics education (mainly in secondary school), to the "relativist ontology" that is advocated by radical constructivism; here the appearance of unphysical solutions does not pose an epistemic problem.