In light of the replication crisis in psychology, null-hypothesis significance testing (NHST) and "p"-values have been heavily criticized and various alternatives have been proposed, ranging from slight modifications of the current paradigm to banning "p"-values from journals. Since the physics education research community often relies on quantitative statistical approaches, the challenges the replication crisis poses to these approaches need to be considered. "p"-values suffer primarily from the fact that they carry little information by themselves and lend themselves to misinterpretations. As one alternative, Bayesian approaches have become increasingly popular as the posterior distributions they provide carry more relevant information than "p"-values. In this paper, we discuss practical issues related to "p-values" with respect to interpreting and communicating results and how these issues can be addressed using a Bayesian approach. Drawing on a science education data set, we demonstrate how Bayesian data analysis methods go beyond p-values and can help to make more valid conclusions and to communicate them more easily in a manner that lends itself to less misinterpretations.