Teaching poetry presents difficulties of various kinds, and these can be illustrative of wider curricular and pedagogical challenges. This paper considers questions of knowledge and remembrance as they arise in two poems--Paul Celan's "Todesfuge" and Seamus Heaney's 'Mid-Term Break'--placing these in relation to Jean-François Lyotard's conception of the immemorial. Lyotard is concerned with a kind of repression in dominant conceptions of knowing and thinking, and with the insulation from experience that this effects. I shall argue that, in reading the poems, it is not so much the seeking of essential meanings that is at stake as a turning of attention to what cannot be fully grasped or contained. Towards the end, and with reference to Stanley Cavell, I discuss how refinement of judgement in relation to works of art can lead the way in shaping responsibility in our ethical and political lives. Such responsibility will be evident in our very words where these presuppose, depend on and are addressed to those of others.