Mass adoption of neural machine translation (NMT) tools in the translation workflow has exerted a significant impact on the language services industry over the last decade. There are claims that with the advent of NMT, automated translation has reached human parity for translating news (see, e.g. Popel et al. 2020). Moreover, some machine translation (MT) research has already been done in the context of literary texts. In this paper, we share the results of a pilot study carried out with two groups (a pre-course group and post-course group) of MA-level students participating in a course that involved translating culture-bound texts. The students' role was to post-edit and evaluate two machine-translated stories (Polish legends), marking their comprehensibility and accuracy. We discuss the lessons learnt during this pilot study, the critical errors detected by the students and their perceptions of the end products and the experiment itself. We report noticeable differences found between the pre-course group and the post-course group in terms of language awareness and the speed and quality of their post-editing (PE) performance. Our results also show that the task of post-editing culture-bound texts offers students a unique and enjoyable setting, enabling them to assess translation technology and hone their translation skills at the same time.