We studied the impact of introducing first-year computer science (CS) students to ethical thinking about the social justice impacts of data collection, tracking, bias, internet privacy, and competitive "real world" system design and critique activities. While basic content was consistent for all, one group was involved throughout the course in peer discussions designed to foster greater engagement, with the anticipation that this would enable students to reach new levels of sensitivity through peer-to-peer interaction. This article reports on our observation of this design, interview, and project data collected throughout the course as well interviews conducted eight months later to learn about how students were retaining and applying what they learned. We found that students are sensitive to the technology-related risks and vulnerabilities encountered by individuals based on race, gender, and, to some extent, age, but they struggle to assess who is responsible for these risks, what to do about bias in technology design, and how to mitigate harms for individuals whom they perceive to be vulnerable, furthering the argument for an integrated ethics curriculum. We explore the value of formal peer-led discussion to evolve social justice thinking with a focus on identity, though note that opportunities for any group discussion are meaningful to students' thinking about social justice. Over the longer term, students tend to recall and apply ethics that is closely related to their identity, suggesting that empathy has limits.