Purpose: The current study applies an innovative approach to the assessment of second language listening comprehension skills. This is an important focus in need of innovation because scores generated through language assessment tasks should reflect variation in the target skill and the literature broadly suggests that conventional methods of assessing listening may fall short of achieving this principle (Field [2019]. "Rethinking the Second Language Listening Test from Theory to Practice." Equinox). Design: The study investigated interactions between different methods of presenting listening comprehension questions, the focus of the comprehension questions and the relative authenticity of the sound file in an English-medium university entrance listening test. In a balanced design, 61 participants completed a listening test featuring both scripted and unscripted sound files by answering explicit and implicit information comprehension questions under five counterbalanced question preview and presentation conditions. Test scores were analysed using ANOVA and examined for interactions. Findings: The results revealed interesting relationships between text authenticity and item focus whereby item responses were most frequently correct overall on the explicit items on scripted tasks. However, the reverse was observed on the implicit items, which were more frequently correct on the unscripted tasks. Value: The research findings have important implications for construct definition and highlight possible directions for the development of pedagogical listening tasks and assessments for English-medium academic study.