The limits of AI semiotics: A pilot study probing generative AI image models’ understanding of causality and abstraction

Maja Bak Herrie, postdoc, Department of Art History, Aesthetics & Culture and Museology, Aarhus University
Simon Aagaard Enni, consultant, the Statistics and Machine Learning Team, Danish Technological Institute

Abstract: AI image generation models show great promise in simulating high quality imagery, yet they also tend to fail in subtle and strange ways: producing hands with 6 fingers or text that resembles no language. We believe, that there are foundational limitations to the way state-of-the-art AI image generation models respond to different types of visual signification, i.e., in their understanding of the relation between the prompt and what that prompt means. In order to probe these AI model limitations, we apply the semiotics of Charles S. Peirce, specifically his tripartite model of signs. In pilot study, we investigate how successfully different image generation models respond to prompts relying on each of the three different types of signification in Peirce’s model and trace the results back to the underlying techniques used to make the models. The results of our experiments, whether positive or negative, open up new questions of the depth or shallowness of AI image generation models’ understanding of signs.

Synergy and Synchronization in Dynamic Social Interaction: A Lindy Hop Partner Dancing Case Study

Peter Thestrup Waade, PhD student, Interacting Minds Centre, Aarhus University

Abstract: In experimental research on joint action and coordination, synchrony (i.e., similar relative temporal ordering of actions) is often used as an operationalization of coordination, which then can be related to measures of joint agency. This is appropriate in simple, goal-directed contexts where synchrony is the explicit goal. In this project, however, we demonstrate that research on joint action and dynamic social interaction can move beyond these experimental and measurement limitations, using improvised partner dancing (Lindy Hop) as a naturalistic and goal-free, physically measurable, activity.

In this study, we introduce synergy (i.e. the degree to which coupled systems form an emergent whole, with greater predictive information than is contained in the constituent parts) as an important quality of movement coordination, which can be operationalized with tools from information theory. We investigate the claim that dancing is a synergetic activity, and the relationship between synchrony, synergy, and a sense of joint agency. We also investigate how distributedness of leader-follower dynamics modulate these relationships. This provides an exciting new possibility for quantifying various aspects of social interaction.

Free of charge - All are welcome