Given the widened access to Chinese higher education, a large proportion of students studying at Chinese colleges come from rural lower-class families and encounter different challenges in English language learning. By using Bourdieu's social theoretical lenses, this case study explores how the interplay of capital, habitus and social fields affected ten rural lower-class college students' English language learning. This study finds that these students inhabited difficult and inferior positions because they lacked different forms of capital needed for accessing language and cultural learning resources, solidifying their English language foundation, and developing intercultural understanding and competence. Their narrow, test-oriented focus, and the lack of confidence and initiative in English language learning stemmed from their habitus formed in their situated socio-cultural and educational fields. Through reflecting on these complexities and conditions in English language learning, teachers and rural lower-class college students may move from a deficit lens in terms of their knowledge foundation and learning styles, which leads to the lack of confidence in teaching and learning; instead, they might collaboratively explore appropriate strategies and resources that cultivate students' comprehensive English language competence and intercultural competence.