This study addresses the applicability of generative artificial intelligence (GenAI) within the administrative learner support services at Anadolu University Open Education System, a giga university with more than one million learners in Türkiye. The study reveals the performance differences between a rule-based chatbot and different GenAI-based chatbots using a qualitative case study approach. Learner inquiries that the rule-based chatbot could not answer, and frequently asked questions (FAQs) were posed to ChatGPT and Bing Copilot applications in four different cases. The 708 answers in these scenarios were evaluated by three experts. It was observed that GenAI matched learner questions more effectively than the rule-based chatbot. Additionally, Bing Copilot was more successful in generating responses to learners' questions from the internet compared to ChatGPT, which utilized the FAQ dataset. The study reveals that Gen-AI based chatbots that can work together with rule-based chatbots are more successful in generating answers. Findings demonstrate the potential of GenAI to provide learners with continuous, more natural, and personalized support. These findings offer significant insights into how administrative support services in mass-scale educational institutions can be transformed.