The role of metalinguistic knowledge in learning a foreign/second language is the subject of continuing debate. In this paper we report on a study which explores this issue further with reference to undergraduate students of Chinese as a second/foreign language. The first part of the paper describes the process of developing and validating an instrument designed to elicit information about students' knowledge of the Chinese grammatical system via a pilot study involving undergraduate students of Chinese enrolled in intermediate-level courses at an Australian university. In the second part of the paper, we report on the relationship between various aspects of Chinese grammatical knowledge (as measured by this instrument) and learners' subsequent performance on both continuous assessment tasks and on end-of-semester Chinese language examinations. Findings reveal that while knowledge of the Chinese grammatical system is a good predictor of course performance overall, some aspects of this knowledge are more critical than others, and the knowledge-proficiency relationship varies in strength according to the nature of the assessment task and learners' prior experience of language study. Possible explanations for these findings are offered and suggestions put forward as to how the test described here might be used in future research.