This article compares second language test-takers' performance on an academic listening test in an audio-only mode versus an audio-video mode. A new method of classifying video-based visuals was developed and piloted, which used L2 expert opinions to place the video on a continuum from being content-deficient (not helpful for answering comprehension items) to content-rich (very helpful for answering comprehension items). The video for one testlet contained only the speaker's non-verbal cues and was found to be content-deficient. The other video contained non-verbal cues overlapping with PowerPoint text and was deemed content-rich. Seventy-three ESL learners participated in the study. The video type classification method was shown to be reliable and practical. The results of the Rasch analysis showed no significant impact of condition, either the content-deficient or the content-rich, either at the testlet level or at the item level. Possible reasons and implications of these findings are discussed.