Research-based assessments (RBAs) allow researchers and practitioners to compare student performance across different contexts and institutions. In recent years, research attention has focused on the student populations these RBAs were initially developed with because much of that research was done with "samples of convenience" that were predominantly white men. Prior research has found that the Force Concept Inventory (FCI) behaved differently for men and women using differential item functioning (DIF) analysis. We extend this research in two ways. First, we test the FCI for DIF across the intersection of gender and race for Asian, Black, Hispanic, White, and White Hispanic men and women. Second, we apply the Eaton and Willoughby five-factor model of the FCI to interpret the results of the DIF analysis. We found large DIF on a large number of FCI items. The patterns of items with large DIF follows the five-factor model. The alignment of DIF with this factor structure, along with the measurement invariance of this factor structure across these ten social identities, indicates that the items on the FCI are likely not biased but are instead measuring real differences in physics knowledge among these groups. We frame these differences as educational debts that society owes to these marginalized groups that physics instruction needs to actively repay.