Artificial intelligence (AI) is becoming ever better and more powerful, which is why it can be found in many areas of everyday life. AIs like ChatGPT have also found their way into universities and more and more students are trusting them. Previous studies found opportunities of ChatGPT in academic education such as personalized and interactive learning. But also, risks have been revealed such as cheating in tests or questionable quality of responses. This article examines possible effects of ChatGPT on the learning success of bachelor students, concentrating on following aspects of learning success: (1) are the test scores of ChatGPT-based self-studies different from traditional lectures, (2) can ChatGPT-based self-study accelerate the learning process, and (3) are female students disadvantaged by the use of ChatGPT. In order to answer these questions, the research design draws on experiments that were carried out in two courses at {German University} with a total of 68 participants. The experimental design considered a random assignment of students into two groups: (1) a traditional professor-led lecture group and (2) a self-study group that uses only ChatGPT. The learning success was examined by a learning goal-oriented single-choice test. In addition, the time required for the self-study unit was collected and analyzed. Based on the raw results, including the gender of the participants, independent t-tests have been applied to test the research hypotheses. Among others, it turned out that the test results do not differ significantly between the traditional lecture and the ChatGPT self-study groups, where the ChatGPT groups required significantly less time for preparation. With regard to gender, the study found no evidence of disadvantages for female students. Limitations in this project include the short excerpt of the learning process and differences in teaching styles, the possible effects of which were not taken into account. It is recommended that these topics be explored in future research projects.