The treatment of artificial intelligence (AI) in the field of education has so far been typically characterized by (a) information about how AI can assist educators in carrying out their work, and (b) concerns about the misuse of AI by learners, for example, concerning plagiarism. The links between AI and ethics within the field of education are much more complex. Beyond the concerns about the organization of teaching and learning with the rise of AI--and the associated rights to privacy and safety--there are legitimate needs for instructors and learners to understand how AI affects their daily lives. What are the wider ethical considerations for using AI, particularly from the perspective of human rights norms? This paper critically analyzes some of the human rights at stake regarding the use of AI and its implications for the organization and content of formal education (K-12 and higher education). The human rights perspective on AI's dynamic and changing field--AI human rights literacy--is critical to convey to instructors and learners as they navigate these new technological developments. This paper overviews human rights relevant to everyday encounters with AI in education. It proposes an AI Human Rights curriculum to help both learners and educators become critically aware of these human rights implications.