Abstract

Numerous studies have explored crossmodal correspondences, yet have so far lacked insight into how crossmodal correspondences influence audiovisual emotional integration and aesthetic beauty. We studied the behavioral and neural underpinnings of crossmodal emotional congruency on art perception. In the experiment, the paintings were presented without music (unimodal condition) or with music (crossmodal condition). In the crossmodal condition, the music could be emotionally congruent (e.g., happy painting, happy music) or incongruent with the painting. We included Fourier scrambled versions of the paintings to control for the effect of semantics. We tested 21 participants by using naturalistic art stimuli with fMRI while they rated the presentations. Beauty ratings were higher in the emotionally congruent condition compared to the incongruent condition, with a significant interaction effect with semantics.  This suggests that semantic information at congruent trials enhances the beauty ratings. The fMRI results showed that the crossmodal-unimodal contrast predominantly activated auditory and emotion-processing regions, including the Heschl Gyrus, Superior Temporal Gyrus, and Insular Cortex. Further,  congruent and incongruent presentations showed marked differences in related sensory areas. For incongruent versus congruent comparisons, the results suggest that a frontoparietal network and the caudate may be involved in emotional incongruency. Congruency × Semantics revealed heightened activation in emotion-related regions, specifically the Right Putamen and Amygdala. Moreover, emotional congruency enhanced in-depth processing, which was observed a stronger activations in the ventral stream compared to emotional incongruency. Overall, these findings emphasize the complex connection between emotional congruency and semantic processing in audiovisual affective integration. 

About the speaker

Funda Yilmaz, PhD candidate, Donders Institute for Brain, Cognition and Behaviour, Radboud University

Free of charge - All are welcome