In the present study, I examined the effects of rater characteristics, in particular, raters' familiarity with a foreign accent, on the assessment of second language (L2) pronunciation. Forty-three native English-speaking teachers were divided into three groups according to their reported types of familiarity with Korean accents: heritage, familiar, and unfamiliar. The teachers assessed global proficiency and accentedness of speech samples provided by 40 Koreans speaking L2 English. The ratings made on 1-9 numerical scales were analyzed for interrater reliability and rater severity. While the three teacher groups overall exhibited high interrater reliability, the findings suggest that prior exposure to, or regular contact with, foreign-accented speech may affect teachers' consistency in ratings. A comparison of the three groups' severity measures revealed that the heritage teachers were more lenient than the non-heritage (i.e., familiar and unfamiliar) teachers in evaluating global proficiency and accentedness. The difference in the level of severity between the familiar and unfamiliar teacher groups was not significant.