Predictive modeling is a core technique used in tackling various tasks in learning analytics research, e.g., classifying educational forum posts, predicting learning performance, and identifying at-risk students. When applying a predictive model, it is often treated as the first priority to improve its prediction accuracy as much as possible. Class balancing, which aims to adjust the unbalanced data samples of different class labels before using them as input to train a predictive model, has been widely regarded as a powerful method for boosting prediction accuracy. However, its impact on algorithmic bias remains largely unexplored, i.e., whether the use of class balancing methods would alleviate or amplify the differentiated prediction accuracy received by different groups of students (e.g., female versus male). To fill this gap, our study selected three representative predictive tasks as the testbed, based on which we (1) applied two well known metrics (i.e., hardness bias and distribution bias) to measure data characteristics to which algorithmic bias might be attributed; and (2) investigated the impact of a total of eleven class balancing techniques on prediction fairness. Through extensive analysis and evaluation, we found that class balancing techniques, in general, tended to improve predictive fairness between different groups of students. Furthermore, class balancing techniques (e.g., SMOTE and ADASYN), which add samples to the minority group (i.e., oversampling) can enhance the predictive accuracy of the minority group while not negatively affecting the majority group. Consequently, both fairness and accuracy can be improved by applying these oversampling class balancing methods.