The global COVID-19 health pandemic caused major interruptions to educational assessment systems, partially due to shifts to remote learning environments, entering the post-COVID educational world into one that is more open to heterogeneity in instructional and assessment modes for secondary students. In addition, in 2020, educational inequities were brought to the forefront of social conscience. The purpose of this study is to empirically explore how contextual (i.e., school-level) race and economic factors may relate to and explain measurement challenges that can arise during shifts to remote learning. We fit a series of multilevel models to explore school-level factors in assessment data alongside psychometric problems of differential item functioning and person fit in classroom assessment measurement models. Our results demonstrate ways in which our project's classroom assessments were impacted by shifts to remote learning, emphasizing the importance of researchers and practitioners evaluating such concerns when seeking validity evidence for interpretation of classroom assessment data.