The approaches of "problem-based learning" and "writing to learn" are known for facilitating the apprehension of concepts and better retaining of knowledge. In educational research, concept maps are sometimes used to assess the learners' level of knowledge. In this paper, the main aim is to investigate the validity of concept maps as an instrument for the assessment of learning. Therefore, six students were observed for more than a year and their learning process was documented in various ways. The concept maps were used in the form of a pre-post-test, and the different students' results were compared in a cross-case analysis using a master concept map. The results presented in this study indicate that the validity of concept maps compared to interviews and reports are questionable. It is possible to measure some parts of the learning process with concept maps, but conceptual learning seems to be hidden from the instrument. Therefore, concept maps might not be the most useful tool to measure conceptual change.