The paper emphasises the need to consider the social, political, and economic contexts within which educational systems operate and use Artificial Intelligence (AI). Focused on Swedish compulsory education, this study explores whether the use of AI envisioned by national authorities and educational technology companies contributes to unfairness. Through qualitative content analysis of Swedish policy documents and educational company reports, the study employs the concept of Relevant Social Groups to assess how diverse stakeholders perceive the risks and benefits of using AI in education regarding fairness. Three distinct groups are identified, each prioritising different forms of "efficiency" as a key value--economic, pedagogical, and accessibility-related. In conclusion, this study sheds light on the intricate interplay between fairness and the use of AI in the Swedish educational system. It also questions the concept of fairness revolving around formal equality of opportunities that separates fairness from the broader context of social justice.