The advent of COVID-19 changed how education is carried out at the tertiary level in Ghana. Despite the lunch of emergency remote teaching at the University of Ghana in 2020, little research has been done on students' experiences. Using a qualitative descriptive design, this study explored students' experiences regarding the benefits, challenges, and the influence of emergency remote teaching on their self-esteem, academic confidence, and performance. 20 students were interviewed, and their responses were analyzed using thematic analysis. Results showed students had both positive and negative experiences which had diverse impacts on their self-esteem, academic confidence, and performance. Positive experiences resulted from the confidence to participate in class without intimidation, improved IT literacy, self-esteem and academic performance, less distraction and leniency from lecturers. Negative experiences bothered on problems with the learning management system, internet challenges, laziness on the part of students, high cost of internet data, unconducive learning environment, comprehension difficulties, increased workload and cheating by students. The findings underscore the need for university management to consider students' psychological wellbeing and needs when implementing new educational systems and further put measures in place to improve emergency remote teaching and curb academic dishonesty in order to improve tertiary education in Ghana amidst COVID-19.