The current study examined the degree to which collocation use (i.e., meaningful co-occurrences of multiple words) is related to first language (L1) raters' intuitive judgments of second language (L2) speech. Speech samples from a picture description task performed by 85 Japanese learners of English with varied L2 proficiency profiles were transcribed for 10 L1 raters to access for global comprehensibility (the extent to which speech can be easily understood) and lexical appropriateness (the extent to which words are used adequately and naturally in context). The samples were then submitted to a range of lexical measures tapping into the collocation (frequency, association), depth (abstractness), and breadth aspects (frequency, range) of L2 vocabulary use. Results of the statistical analyses showed that the raters' comprehensibility and lexical appropriateness scores were strongly determined by the L2 speakers' use of low-frequency combinations containing infrequent, abstract, and complex words (i.e., mutual information).