Pre-recorded lectures have become a prevalent approach in online education due to the proliferation of MOOC platforms and the COVID-19 pandemic. However, due to the lack of real-time interactions between instructors and learners, learners have encountered various difficulties in understanding the lectures and actively engaging with the learning materials. Therefore, various attempts have been made to aid learning by revealing the cues of the instructor, including countenance, digital pointer, and gaze, in video lectures. Nevertheless, the current body of research still lacks a comprehensive exploration of these non-verbal cues, critical for understanding and enhancing the learning experience. Consequently, recent studies on visualization even with novel cues often fail to match the learners' needs. Therefore, in this paper, we explore the impact of instructors' non-verbal cues, focusing on the countenance, digital pointer, and gaze, in video lectures and introduce a novel gaze visualization called "Elements-aware Gaze." First, we determine learners' design preferences and optimize design parameters for "Elements-aware Gaze." Afterward, we analyze the usability, workload, and cognitive load of non-verbal cues in video lectures through a user study with 37 subjects. The findings highlight the practicality of the "Digital Pointer" and the effectiveness of "Elements-aware Gaze," recognized for its utility and engaging learning experience while minimizing the workload in video lectures. Finally, our results emphasize the importance of thoughtful design and strategic integration of non-verbal cues in video lectures, providing a valuable foundation for enhancing remote learning experiences in the evolving online education landscape.