Open-ended assessments, defined as assessments with a large set of possible correct answers, by nature lend themselves to concerns regarding accurate and consistent grading. This article describes one particular open-ended assessment, named Creative Exercises (CE), designed for promoting students' interconnection of concepts in a college general chemistry setting. The article presents evidence concerning several aspects of validity, including the extent scores represent chemistry knowledge and the extent scoring is consistent across three graders. The evidence is also presented in the context of what is known about concept maps, a commonly employed open-ended assessment in chemistry. Implications for the administration of CEs and the appropriateness of measuring students' hierarchical organization of knowledge are also discussed as a result of this comparison. (Contains 4 tables.)