University rankings have gradually become an issue for concern in the academic community worldwide. Several mechanisms with different methodologies have been developed to rank the universities appropriately. However, some ranking tools have notable issues, especially with the indicators adopted. Some are based merely on research performance, whilst others have focused solely on specific fields, such as science and technology -- which could have deprived those in the arts and social sciences. This paper uses a narrative review to highlight a number of inconsistencies in the methodologies applied to rank universities. Five main ranking tools commonly applied to the world's universities are reviewed, namely "Quacquarelli Symonds" (QS), "Times Higher Education" (THE), "Academic Ranking of World Universities" (ARWU), "Leiden University" ranking and "Webometrics" ranking. We found that several flaws in the rankings caused inconsistencies in university placings in different rankings. Suggestions for integrating multiple criteria and indicators for better ranking exercises are proposed.