Online discussion forums have become an essential part of university courses, whether the course is conducted online, or face to face, or in mixed or blended mode. Discussion forums are considered to engage students better with the course content and encourage them to share and gain knowledge from each other. However, online engagement does not always happen automatically between students. Hence grading of discussion forum participation has been recommended to ensure quality student participation. Currently, a major focus has been put onto the better use of discussion forums, but the way in which the quality of participation can be evaluated has yet to be adequately investigated. Furthermore, evaluation of the instructor participation in a discussion forum and its impact on students and their contributions is lacking. In this paper, we report on our research into online discussion forum quality through analysis of discussion forum activities, along with student focus group meetings and instructor interviews. We have devised a set of criteria for evaluating discussion forum activities. Our results show that students depend highly on the instructor's feedback and the participation of the students can only be evaluated with reference to the moderation of the instructors. (Contains 2 tables.)