Theories of visual narrative comprehension have advocated for a hierarchical grammar-based comprehension mechanism, but only limited work has investigated this hierarchy. Here, we provide a computational framework inspired by computational psycholinguistics to address hierarchy in visual narratives. The predictions generated by this framework were…

Multi-modal discourse comprehension requires speakers to combine information from speech and gestures. To date, little research has addressed the cognitive resources that underlie these processes. Here we used a dual-task paradigm to test the relative importance of verbal and visuospatial working memory in speech-gesture comprehension. Healthy,…

Seeing pictures of objects activates the motor cortex and can have an influence on subsequent grasping actions. However, the exact nature of the motor representations evoked by these pictures is unclear. For example, action plans engaged by pictures could be most affected by direct visual input and computed online based on object shape.…

We have developed a metabolic pathways visualization skill test (MPVST) to gain greater insight into our students' abilities to comprehend the visual information presented in metabolic pathways diagrams. The test is able to discriminate students' visualization ability with respect to six specific visualization skills that we identified as key to…

The current paper explores possible solutions to the problem of detecting affective states from facial expressions during text/diagram comprehension, a context devoid of interactive events that can be used to infer affect. These data present an interesting challenge for face-based affect detection because likely locations of affective facial…

Cantonese shape classifiers encode perceptual information that is characteristic of their associated nouns, although certain nouns are exceptional. For example, the classifier "tiu" occurs primarily with nouns for long-narrow-flexible objects (e.g., scarves, snakes, and ropes) and also occurs with the noun for a (short, rigid) key. In 3…

Purpose: The purpose of this paper is to examine if university students could accurately extract information from graphs presented in 2D or 3D formats with different colour hue variations or solid black and white. Design/methodology/approach: Participants are presented with 2D and 3D bar and pie charts in a PowerPoint presentation and are asked to…