Our study focuses on the correlation of concept map (CMap) structures and learning success tested with short answer tests, taking into particular account the complexity of the subject matter. Novice sixth grade students created CMaps about two subject matters of varying difficulty. The correlation of the complexity of CMaps with the post-test was small but highly significant in both subject matters. The complexity of the CMaps correlated with the long-term knowledge in the difficult subject matter but not in the context of the easy one. Furthermore, the high number of technical errors makes it close to impossible to estimate students' knowledge. In summary, CMaps do not provide an adequate alternative to conventional short answer knowledge tests, but together with them they may offer a better comprehension of a student's knowledge structure and aid in the preparation of further instruction tailored to individual needs. (Contains 4 tables and 4 figures.)