Lexical prosody (e.g., stress and pitch accent) has been shown to constrain lexical activation of spoken words in various languages. In the present study, whether or not the constraint of lexical prosody is affected by word familiarity in lexical access of Japanese words was examined using a cross-modal priming task. The stimuli were pairs of prosodically different homophones (minimal accent pairs). When the targets were more familiar members of minimal accent pairs, the responses were facilitated by prior presentations of primes that were prosodically different homophones of the targets, suggesting that lexical prosody did not constrain lexical activation. In contrast, when less familiar members of minimal accent pairs were used as the targets, the prosodically different homophones did not facilitate the responses to the targets. These results suggest that the constraint of lexical prosody is not so strong but is affected by the factor of word relative familiarity.