Background: A growing body of literature has demonstrated the importance of discourse assessment in patients who suffered from brain injury, both in the left and right hemispheres, as discourse represents a key component of functional communication. However, little is known about the relationship between gray matter density and macrolinguistic processing. Purpose: This study aimed to investigate this relationship in a group of participants with middle-low to low socioeconomic status. Method: Twenty adults with unilateral left hemisphere (n = 10) or right hemisphere (n = 10) chronic ischemic stroke and 10 matched (age, education, and socioeconomic status) healthy controls produced three oral narratives based on sequential scenes. Voxel-based morphometry analysis was conducted using structural magnetic resonance imaging. Results: Compared to healthy controls, the left hemisphere group showed cohesion impairments, whereas the right hemisphere group showed impairments in coherence and in producing macropropositions. Cohesion positively correlated with gray matter density in the right primary sensory area (PSA)/precentral gyrus and the pars opercularis. Coherence, narrativity, and index of lexical informativeness were positively associated with the left PSA/insula and the superior temporal gyrus. Macropropositions were mostly related to the left PSA/insula and superior temporal gyrus, left cingulate, and right primary motor area/ insula. Discussion: Overall, the present results suggest that both hemispheres are implicated in macrolinguistic processes in narrative discourse. Further studies including larger samples and with various socioeconomic status should be conducted.