A growing body of research and popular reporting shows racial differences in school modality choices during the COVID-19 crisis, with White students more likely to attend school in person in the fall of 2020 and spring of 2021. This in-person learning gap raises serious equity concerns. We use unique panel survey data to explore possible explanations. We find that a combination of factors may explain these differences. School districts' offerings, political partisanship, perceived risk from the pandemic, and local COVID-19 outbreaks are all meaningfully associated with and plausibly explain the in-person learning racial gap. Our results illustrate how not only policy decisions but also political leanings and individuals' beliefs could contribute to inequality in access to learning and illustrate the need for a better understanding of the factors behind observed racial inequalities in education.