Many research-related classes in social sciences present probability as a unified approach based upon mathematical axioms, but neglect the diversity of various probability theories and their associated philosophical assumptions. Although currently the dominant statistical and probabilistic approach is the Fisherian tradition, the use of Fisherian significance testing of the null hypothesis and its probabilistic inference has been an ongoing debate. This paper attempts to explore the richness and complexity of the ideas of probability with the emphasis on the relationships between Fisherian and other probability theories. First, it clarifies the differences between Fisher and Jeffreys and explains the background history relating to Fisher's quest for certainty. Second, it explains the differences between Fisher and Pearson and explains the limitations of the Fisherian approach. In addition, it argues that although Fisher criticized the Bayesian school for its alleged lack of objectivity, Fisher's quest for certainty is driven by his subjective faith in experimental methods, eugenics and Darwinism. Last, it will briefly introduce the synthesized approaches by Berger and Pawitan, respectively, as a possible remedy. (Contains 2 figures.)