Universities are recognizing the need to prepare graduates to think conceptually and have the ability to take on complex, real-world problems. Strategies to assess conceptual knowledge are limited and often require more time and effort to complete than is accessible for most undergraduate courses. Card sorting is a very broad technique for understanding how people group concepts, but in higher education has typically been used to show a student's development towards expert-like thinking in a discipline as a whole. However, it typically does not give much insight into how we should change our teaching. In this paper, using the novel setting of two terms of a first-year, earth and ocean science lab that uses problem-based learning (PBL), we show how one can generate a card sort that is built using course learning goals and then use the analysis to make actionable improvements to course instruction. Using a card sort designed so that the expert sort corresponds to learning goals supported by the lab activities, we found that in both offerings of the course students generally moved towards expert-like sorting with a reduction in novice-like sorting. A striking feature stood out in both terms of the course, with one question scoring significantly lower than any other expert pairings, despite a change in the wording of that question between terms. This suggests that our course materials do not promote this specific conceptual connection that we had expected and gives us a clear place to look for issues in our course material. In a broader context, our results suggest that tailoring card sort questions to material at a course level, rather than at the discipline level, can provide a manageable, routine assessment of conceptual knowledge in students, while also providing feedback on the quality of course materials.