Communicating science effectively is an essential part of the development of science literacy. Research has shown that introducing primary scientific literature through journal clubs can improve student learning outcomes, including increased scientific knowledge. However, without scaffolding, students can miss more complex aspects of science literacy, including how to analyze and present scientific data. In this study, we apply a modified CREATE(S) process (Concept map the introduction, Read methods and results, Elucidate hypotheses, Analyze data, Think of the next Experiment, and Synthesis map) to improve students' science literacy skills, specifically their understanding of the process of science and their ability to use narrative synthesis to communicate science. We tested this hypothesis using a retrospective quasi-experimental study design in upper-division undergraduate courses. We compared learning outcomes for CREATES intervention students to those for students who took the same courses before CREATES was introduced. Rubric-guided, direct evidence assessments were used to measure student gains in learning outcomes. Analyses revealed that CREATES intervention students versus the comparison group demonstrated improved ability to interpret and communicate primary literature, especially in the methods, hypotheses, and narrative synthesis learning outcome categories. Through a mixed-methods analysis of a reflection assignment completed by the CREATES intervention group, students reported the synthesis map as the most frequently used step in the process and highly valuable to their learning. Taken together, the study demonstrates how this modified CREATES process can foster scientific literacy development and how it could be applied in science, technology, engineering, and math journal clubs.