Inter-brain dynamics of interpersonal synchrony during continuous auditory-motor rhythm production

Anna Zamm, assistant professor, Department of Linguistics, Cognitive Science and Semiotics, Aarhus University

Abstract: Interpersonal synchrony -the temporal coordination of actions between individuals - is fundamental to many human social behaviours, from group music-making to team sports. Emerging evidence from cognitive neuroscience suggests that interpersonal synchrony arises from inter-brain coupling of partners' cortical sensorimotor activity. One open question is how inter-brain coupling contributes to continuous dynamics of interpersonal synchrony during ongoing interaction. The proposed EEG hyperscanning project addresses this question using a novel joint musical rhythm production task. Pairs of human subjects will synchronously produce circular rhythmic motions on a circular touch-sensor; partners' touch locations on the sensor will be turned into sound (sonified) such that partners can synchronize movement via auditory feedback, as typically occurs during group music-making. EEG will be simultaneously recorded from partners  Partners' behavioural synchrony will be assessed by computing the Continuous Relative Phase (CRP) of partners' movement on the touch sensor. Partners' inter-brain synchrony will be assessed by computing CRP of partners' cortical oscillatory activity in frequency bands typically associated with sensorimotor processes. Most critically, behavioural and neural CRP dynamics will be correlated to assess the fundamental question of how inter-brain coupling contributes to the continuous dynamics of interpersonal synchrony during ongoing interaction.

When will the silence end? – Identifying neural markers of how partners coordinate pauses in social interaction

Anna Zamm, assistant professor, Department of Linguistics, Cognitive Science and Semiotics, Aarhus University

Abstract: Pauses are anubiquitous feature of social interaction: Speech partners pause between turns, musical partners pause expressively between phrases. Here we address the behavioural and neural mechanisms by which groups coordinate the duration of pauses in social interaction, using group music-making as a model of social interaction. Specifically, groups of trained musicians will produce rhythmic sequences that contain unmeasured pauses that they must synchronously resolve. EEG will be measured simultaneously from all group members using a novel hyperscanning platform from AU Engineering. Behavioural and neural data will be assessed to address the hypothesis that the ability to synchronously resolve pauses in social interaction is associated with predictive processes reflected in modulations of cortical sensorimotor oscillations that occur during pauses. 

The taste of cooperation – Disentangling bottom-up versus top-down influences of shared food experience on social affiliation

Qian Janice Wang, assistant professor, Department of Food Science, University of Copenhagen

Abstract: Sharing food is a culturally universal bonding experience. Emerging evidence suggests that eating the same food, or even sharing from the same plate, can promote trust and cooperation between strangers. However, the sensory and cognitive mechanisms by which food sharing facilitates social affiliation are still unknown. The present project aims to disentangle sensory (shared food experience) from cognitive (knowledge of sharing) contributions to social outcomes of food sharing. Two lab-based food-sharing studies will be conducted where, by manipulating what participants are told about the shared foods and what they actually eat, we can disassociate the cognitive knowledge of food-sharing from the sensory experience. Partners will subsequently complete a social coordination game that either requires cognitive cooperation (Study 1, economic game) or sensorimotor coordination (Study 2, synchronization of dyadic finger-tapping). Thus, the present project will elucidate how different pathways to social affiliation via food-sharing (sensory versus cognitive) impact coordination across distinct domains of social behavior. 

Voice- and speech-based markers of neuropsychiatric conditions: assessing methodological and ethical foundations for clinical application

Alberto Parola, postdoc, Department of Linguistics, Cognitive Science and Semiotics, Aarhus University

Abstract: Promising ML applications have shown great potential to identify vocal and speech markers of the most important neuropsychiatric conditions (e.g., Hitczenko et al., 2021; Cohen et al., 2021; Corcoran et al., 2020) and to develop systems able to monitor patients' symptoms and assist clinicians during the assessment process. However, these efforts face important limitations: the limited replicability and generalizability of previous results (Parola et al., 2022; Fusaroli et al., 2021), few attempts to explicitly account for the heterogeneity of the disorders (Mittal, 2021), and no concrete translation into effective clinical applications yet. What is critically lacking is an explicit reflection on the risks and limitations of ML applications that can support the development of robust, effective, and ethically grounded translational work. In this project, we will explore avenues to improve the clinical impact of ML applications in speech and voice research, focusing primarily on applicability and ethical concerns.  

To this end, we draw on an already collected large dataset of voice and speech samples from the Danish High Risk and Resilience Study - VIA7 study (Gantriis et al., 2019), which examined 522 children born to parents diagnosed with schizophrenia (SZ) or bipolar disorder (BP). Our goal is to develop conservative (i.e., more robust and generalizable) ML and NLP pipelines to identify vocal and language markers of clinical symptoms in children at high-risk, that can serve as a reference for future studies. In addition, we aim to assess the impact of heterogeneity (e.g., socioeconomic, demographic, and clinical differences) and the presence of potential methodological biases and limitations, and robustly test the reliability of the results against various preprocessing and analytical procedure. Finally, we will explore how ML techniques can concretely support the development of robust, effective, and ethically founded clinical applications, and evaluate how to include from the very design of a study a consideration of risks, limitations, and ethical practices. The final outcome is to provide a first solid effort - both conceptually and methodologically - for the development of better practices in ML, SSP and NLP clinical research.

Free of charge - All are welcome to attend