This study draws on a research project where a model of fiction-based ethics education was developed and put into practice during a school year in five classes in compulsory school, two in grade 5 and three in grade 8. A test was constructed with the purpose of evaluating a multi-dimensional ethical competence. The test was given at the beginning and the end of the school year to the students in the five classes, but also to students in five other classes who received the ordinary form of ethics education. The test was constructed by using tasks from earlier Swedish national tests on ethics education, but in this study new assessment instructions were developed. According to this test, almost no significant differences were found between the classes with fiction-based ethics education and the other classes. However, this contrasts with the fruitfulness of the fiction-based model, as identified through other forms of evaluation. The findings raise questions about whether and how ethics education can be meaningfully tested, which in this article is self-critically reflected on. A concluding discussion on target competence, teaching intervention and ways of evaluation -- in relation to each other -- ends the article.