Information units (IUs) have been proposed as a way of grading students' notes. However, little work has been done to validate and check the reliability of such an approach. This study explores how teacher-researchers (TRs) use information units (IUs) to rate content items in students' listening notes using a rubric. Data were collected from six TRs who graded four students' listening notes. Each TR identified and rated the IUs in the students' notes using a 4-point rubric scoring system before commenting on the notes they had graded. The main aim of this study was to examine if each TR awarded similar IU ratings between the items they graded. The results showed there were inconsistencies between the TRs ratings of the students' notes. However, each TR consistently interpreted the IUs in the same way internally across multiple texts. These findings indicate that further training in using IUs is needed to add validity and reliability to the potential rating system for grading students' listening notes.