Emotion understanding (EU) is a socio-cognitive skill that enables us to understand the expression of emotions in ourselves and others. Exposure to early adversity hinders its development, since quality social interactions are essential for its growth. Language is a critical component of EU, and therefore, it may be a mediator between early adversity exposure and EU. In this study, we analyzed EU development and the mediating role of language in the relation between early adversity and EU in internationally adopted children, maltreated children in residential care, and a community comparison group. Children (N = 145) were between 4 and 8 years old at assessment. A total of 39 internationally adopted children, 48 maltreated children living in residential care, and a community comparison group of 58 children with no contact with child-protective services participated. We assessed EU using the Test of Emotional Comprehension and language using a receptive syntax language assessment. Both internationally adopted and maltreated children in residential care lagged behind community children in EU, with internationally adopted children showing a higher EU than children living in residential care in some areas. The effect of being in the adopted or residential care group on EU was mediated through receptive language, partially in the case of children in residential care. Results are discussed considering the potential for catch-up in socio-emotional development and the importance of adequate statistical modeling of mediator variables.