Science instructors are increasingly incorporating teaching techniques that help students develop core competencies such as critical-thinking and communication skills. These core competencies are pillars of career readiness that prepare undergraduate students to successfully transition to continuing education or the workplace, whatever the field. Course-based undergraduate research experiences that culminate in written research papers can be effective at developing critical-thinking and communication skills but are challenging to implement as class size (and student-to-instructor ratio) grows. We developed a hierarchical mentoring program in which graduate student mentors guided groups of four to five undergraduate students through the scientific process in an upper-level ecology course. Program effectiveness was evaluated by grading final research papers (including previous year papers, before the program was implemented) and surveys (comparing to a course that did not implement the program). Results indicated that primary benefits of hierarchical mentoring were improvements in perceived and demonstrated ability in data analysis and interpretation, leading to a median increase in paper score of ~10% on a 100-point scale. Future directions indicated by our study were a need to incorporate more approaches (e.g., low-stakes writing exercises) and resources into a revised program to improve outcomes for students whose primary language is not English.