Providing students with timely, targeted, and useful feedback regarding their understanding of course topics is generally accepted as a good educational practice. However, when classes are very large there are challenges that prevent many instructors from accomplishing this goal. This study explores the perceived helpfulness to students and the instructor of implementing a relatively new method of automated scoring and feedback in a large section of an organic chemistry course. Prior research has shown this method to be helpful in other STEM classes. In the current study, students in two different offerings of a stand-alone organic chemistry course completed an anonymous survey in which they were asked to provide feedback about their perceptions of the new methodology. The faculty member who taught the course was also asked to respond to a series of questions regarding the feedback system. Both students and the instructor provided favorable comments about the helpfulness of the methodology and the feedback provided by it. The instructor found it helpful for providing individual feedback to students, which had previously not been possible due to the number of students enrolled in the course. Students reported that the feedback helped them to identify course topic strengths and challenge areas and that they planned to study the material differently going forward. The results indicate that this intervention can help improve student understanding of course topics and necessary actions for improving future performance in the course.