This study aims to find out the dominant factors that cause students' difficulties in learning abstract algebra. The factors investigated in this study focused on attitude, motivation, prior knowledge, lecturer performance, learning methods, as well as teaching materials. This study is a descriptive research using a survey method. We involved 30 students of mathematics education department in Riau Kepulauan University, Indonesia. To get the data, we used questionnaire and interview. Based on the questionnaire result analysis, there were three dominant factors causing the students' difficulties in learning abstract algebra. They were students' attitudes towards abstract algebra, prior knowledge, and teaching materials with the percentage of 36.7%, 28.4%, and 23.4% respectively (strong category). Meanwhile, the other three factors -- motivation, lecturer performance, and learning method -- had the percentage of 45.4%, 56.7%, and 43.4% respectively (fair category). These results impact on the finding of dominant factors causing the difficulties in learning abstract algebra. At the end, it could be a reference for the lecturer to improve the learning effectiveness.