Theories of visual narrative comprehension have advocated for a hierarchical grammar-based comprehension mechanism, but only limited work has investigated this hierarchy. Here, we provide a computational framework inspired by computational psycholinguistics to address hierarchy in visual narratives. The predictions generated by this framework were…

Viewers' attentional selection while looking at scenes is affected by both top-down and bottom-up factors. However, when watching film, viewers typically attend to the movie similarly irrespective of top-down factors--a phenomenon we call the "tyranny of film." A key difference between still pictures and film is that film contains…

This study aims to bridge the research gap in the humour comprehension problems of individuals with dyslexia in Chinese culture. We conducted a nonexperimental study to examine the differences between Chinese adolescents with and without dyslexia in visual humour comprehension as well as the group differences in the correlation of visual humour…

Recent work has investigated the origin of infant colour categories, showing pre-linguistic infants categorise colour even in the absence of colour words. These infant categories are similar but not identical to adult categories, giving rise to an important question about how infant colour perception changes with the learning of colour words. Here…

This experiment employed viewing time methods to investigate the effects of individual differences in visuospatial working memory (VWM) on the processing of older adults' bridging inferences in the understanding of visual narratives. The results showed that older adults could make bridging inferences in visual narrative processing, and that VWM…

Children with autism spectrum disorder (ASD) have deficits in audiovisual speech perception and temporal processing. The current study has examined the relationship between the audiovisual speech perception deficits and temporal processing deficits in children with and without ASD. To this end, using the McGurk paradigm, we implemented two…

The comprehension of signed language requires linguistic and visual-spatial processing, such as perspective-taking for correctly interpreting the layout of a spatial scene. However, little is known about how adult second-language (L2) learners process visual-spatial constructions in a signed language that they are studying, including which angles…

Multi-modal discourse comprehension requires speakers to combine information from speech and gestures. To date, little research has addressed the cognitive resources that underlie these processes. Here we used a dual-task paradigm to test the relative importance of verbal and visuospatial working memory in speech-gesture comprehension. Healthy,…

As spatial languages, sign languages rely on spatial cognitive processes that are not involved for spoken languages. Interlocutors have different visual perspectives of the signer's hands requiring a mental transformation for successful communication about spatial scenes. It is unknown whether visual-spatial perspective-taking (VSPT) or mental…

Purpose: The use of sign-supported speech (SSS) in the education of deaf students has been recently discussed in relation to its usefulness with deaf children using cochlear implants. To clarify the benefits of SSS for comprehension, 2 eye-tracking experiments aimed to detect the extent to which signs are actively processed in this mode of…

Visual, as compared to verbal, tasks are often assumed to be easier for individuals with autism spectrum disorder (ASD), but is this true for story comprehension? This study evaluated story comprehension monitoring across visual, listening, and written modalities and assessed predictors in two closely matched groups (age, socioeconomic status,…

The auditory scaffolding hypothesis states that early experience with sound underpins the development of domain-general sequence processing abilities, supported by studies observing impaired sequence processing in deaf or hard-of-hearing (DHH) children. To test this hypothesis, we administered a sequence processing task to 77 DHH children who use…

Fifth graders interpreted the book "Doll Bones" by Holly Black through visual representations from the beginning to the end of the book. Each visual representation was analyzed to determine how students responded. Most frequently, they moved to inferential ways of understanding. Students often visually interpreted emotional plot elements…

Coviewing is a commonly recommended practice, but little is known about how coviewing impacts children's educational media viewing experience. We investigated how coviewing impacts attention and comprehension of educational media, as well as the role of baseline vocabulary in understanding these associations. Eighty-three preschoolers viewed two…

When children interpret spoken language in real time, linguistic information drives rapid shifts in visual attention to objects in the visual world. This language-vision interaction can provide insights into children's developing efficiency in language comprehension. But how does language influence visual attention when the linguistic signal and…