Aarhus University Seal

IMC Bootcamp "Assessing Immersion and Engagement in Media and Real World Experiences"

A new bootcamp in the PLAYTrack series

Info about event

Time

Tuesday 21 June 2016, at 09:15 - Wednesday 22 June 2016, at 16:00

Location

IMC Meeting Room

Organizer

Katrin Heimann under PLAYTrack

Commonly, the terms  "immersion" and "engagement" are used more or less specific to refer to the degree of involvement in an interaction/experience, in the strongest case causing not only pleasure and enjoyment, but also having  positive influence on memory and opinion building, as well as/or possibly productivity/creativity. As such, the cognitive states correlated to this involvement have been of equal interest for scholars as well as practitioners from various disciplines keen on exploring or building on/fostering of these effects. 

Such further investigations and interventions critically depend on tools to qualitatively and quantitatively describe the states in focus. In our may´s bootcamp we have invited 8 speakers that have used/developed different methods serving this purpose ranging from questionnaires to imaging data and from the lab to classroom interventions.

Please note that the first day is mainly focusing on assessing engagement during film experiences, while the second day is widening the focus to other realms (including clinical and learning environments).

 

If you would like to participate, PLEASE SIGN UP HERE (limited places)

 

Below the list of speakers, separated in day 1 and day 2. Program starts always at 9:15, with two talks in the morning and two after lunch. Talks should last not longer then 40 min plus generous discussion time. Lunch is provided (sandwiches) and after the last talk we offer a reception with soft drinks, wine and beer for further discussion. Expected end is between 16:00 and 17:00. If you would like to meet one of the invited speakers apart from the program, please contact katrinheimann@cas.au.dk. Please note also that we ask participants to attend the whole day, to enable discussion across talks. 

 

PROGRAM 

DAY 1 (21.6.16)

9:15 start


Introduction - Katrin Heimann 

 

Katalin Balint  & Brendan Rooney

**Reading the mind in films. The effect of shot scale on the complexity and frequency of viewers' theory of mind responding.** Narrative engagement is one of the most commonly used terms in communication studies in regards to immersive experiences with mediated narratives. Following the original conceptualization of Busselle and Bilandzic, the term reflects on a conscious subjective experience in audience members occurring spontaneously while watching or reading a story. Narrative engagement (or related terms transportation, narrative absorption) is associated with focused attention, increased emotional involvement and embodied involvement, as well as with decreased awareness of the outside world. Narrative engagement is a multi-faceted phenomenon. We focus on one of the most important components of narrative engagement that is audience members’ engagement with fictional characters defined as the adoption of characters’ perspectives, motivations, and mental states. The intensity of character engagement is a good predictor of enjoyment, and other narrative effects, such as narrative persuasion. Conventionally, narrative engagement (as well as character engagement) can be investigated at the level of the subjective experience (e.g., with questionnaires for transportation, identification, etc.), the underlying psychological processes (e.g., attention, memory, empathy) or biological indicators (e.g. brain activity, heart rate, skin conductance, or eye movement behavior). The present talk focuses on theory of mind, defined as the awareness and recognition of mental states (e.g., emotions, cognitions, intentions) in others, which is considered to be a key psychological process underlying character engagement. In other words, audience members need to recognize ongoing emotions and intentions of the characters in order to care for the character and appreciate the significance of the related story events. Theory of mind is closely related to the cognitive processes involved in empathy response. In everyday life theory of mind is a crucial process that enables us to effectively interact with each other. The importance of theory of mind in narrative engagement with movies has been already recognized, however, there is only a limited amount of knowledge on the textual / visual features (as opposed to content related features) that facilitate theory of mind responding in film viewers. In this talk we investigate the effect of shot scale on theory of mind responding to fictional characters. Shot scale, defined as the apparent spatial distance of characters from the camera, is one of the most effective visual devices to regulate the relative size of characters’ faces, the relative proportion of the human figure to the background, as well as to arrange film content according to its saliency. The main assumption of our project is that shot scale in movies can modulate viewers’ theory of mind responding. To test this assumption we conducted two experiments. Participants were randomly assigned to one version of the manipulated material and were asked to fill in questionnaire items on the valence, arousal and dominance of the emotional experience; personal relevance of the story, and theory of mind responding to the character. Theory of mind response was assessed through qualitative data collection (i.e. we collected linguistic data as opposed to numeric or biological markers). In open ended questions participants were asked to describe the story of the film, the perspective of the protagonist and their own feelings. In these free responses the spontaneously occurring theory of mind responding was measured by quantitative content analytic method where independent coders assessed the frequency and complexity of cognitive and affective mental state reasoning, and attribution of intention. In the presentation we discuss the advantages and disadvantages of combining experimental method with qualitative data collection and quantitative content analysis. During the talk we will invite the audience members to participate in and discuss a small experiment illustrative for our topic.

Balint, K., Klausch, T., & Pólya, T. (forthcoming). Watching closely: shot scale influences theory of mind responding in visual narratives. Journal of Media Psychology.

Busselle, R., & Bilandzic, H. (2009). Measuring narrative engagement. Media Psychology, 12(4), 321–347.

Cao, X. (2013). The effects of facial close-ups and viewers’ sex on empathy and intentions to help people in need. Mass Communication and Society, 16(2), 161–178.

Tal-Or, N., & Cohen, J. (2010). Understanding audience involvement: Conceptualizing and manipulating identification and transportation. Poetics, 38(4), 402–418.

Levin, D. T., Hymel, A. M., & Baker, L. (2013). Belief, desire, action, and other stuff: theory of mind in movies. In A. P. Shimamura (Eds.), Psychocinematics (pp. 244–266). Oxford University Press.

 

Stephen Hinde

The Dual Task Instrument for measuring Engagement with Film, what is it measuring?  

Communication research into engagement with mass media has made extensive use of the dual-task paradigm as an instrument for measuring engagement/immersion with media. The methodology was developed within cognitive psychology, based on the limited capacity model of attention, to infer the spare cognitive resource capacity while performing a cognitive task (Kahneman, 1973). However communication researchers found it necessary to produce a more complex model of resource allocation, involving pre-allocation of available resources and their usage in order to interpret their results in a meaningful way (Lang, 2000). Others have questioned this model (e.g. Bezdek, 2012) This study adopted a more fine grained approach to understanding the dual-task paradigm with film. Undergraduate participants watched films and simultaneously used the dual-task instrument. The films were also processed using a computational model processing low-level audio/visual features of the film and parameters linking to the strength of narrative. The model and empirical findings were compared. The study found surprisingly good agreement, between the predictions of the model and the dual- task instrument, with the model predicting over 25% of the variance. Detailed regression analysis revealed that roughly half of this variance in the attentional load was due to low-level audio-visual features; that narrative was important, and that dynamic attention tracked the effects of engagement/immersion in the experience of watching the film stimulus.

Bezdek, M. A. (2012), Changes in Attentional Focus During Suspenseful Film Viewing (PhD Thesis), Stony Brook University, Stony Brook, NY

Kahneman, D. (1973). Attention and Effort. New Jersey: Prentice Hall.

Lang, A. (2000). The limited capacity model of mediated message processing. Journal of Communication, 50(1), 46-70. doi: DOI 10.1111/j.1460-2466.2000.tb02833.x

 

Matt Bezdek 

The engaged brain:Neuroimaging methods to measure the mind at the movies

What can neuroimaging techniques tell us about narrative engagement? One method is to show participants the same film while recording brain activity, and see where synchrony occurs in time or space. Tracking the degree of synchrony across viewers can reveal brain regions with correlated activity patterns, and moments of high synchrony can be tied to narrative features. The strength of neural coherence between viewers may also predict downstream responses in larger real-world populations, such as enjoyment or a propensity to share reactions to narrative content online. A new technique called hyperalignment transforms the spatial pattern of fMRI data recorded during film viewing to align viewers in a common representational space based on the shared narrative experience. Performing hyperalignment significantly increases the spatial extent of intersubject correlation beyond anatomical alignment. Here engagement represents the common processing of the content of a film, but without precise alignment of where the content is represented in the brain. This presentation will provide an overview of these techniques as well as an in-depth look at research I have conducted investigating the dynamic effect of narrative suspense on attention and memory. This research tests a claim of a specific component of engagement: that engaged viewers focus their attention on narrative events and suppress attention to the physical environment. Our behavioral and fMRI evidence suggest that suspense evokes a momentary narrowing of attentional focus, with a suppression of peripheral visual processing. In addition, narrative events that occur at moments of increasing suspense are better recalled than those that occur at moments of decreasing suspense. This finding suggests that the attentional narrowing has downstream consequences for memory encoding and recall. These neuroimaging techniques complement existing behavioral, physiological, and self-report techniques to deepen our understanding of narrative engagement.

References:

Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., & Malach, R. (2004). Intersubject synchronization of cortical activity during natural vision. Science, 303(5664), 1634-1640.

Dmochowski, J. P., Bezdek, M. A., Abelson, B. P., Johnson, J. S., Schumacher, E. H., & Parra, L. C. (2014). Audience preferences are predicted by temporal reliability of neural processing. Nature Communications, 5, 1-9.

Guntupalli, J. S., Hanke, M., Halchenko, Y. O., Connolly, A. C., Ramadge, P. J., & Haxby, J. V. (2016). A model of representational spaces in human cortex. Cerebral Cortex, 1-16.

Bezdek, M. A., & Gerrig, R. J. (2016). When Narrative Transportation Narrows Attention: Changes in Attentional Focus During Suspenseful Film Viewing. Media Psychology, 1-30.

Bezdek, M. A., Gerrig, R. J., Wenzel, W. G., Shin, J., Revill, K. P., & Schumacher, E. H. (2015). Neural evidence that suspense narrows attentional focus. Neuroscience, 303, 338-345. 

 

Monika Suckfuell

The Cognitive and Emotional Processing of Movies

My contribution to the workshop is to describe a research program developed to empirically investigate the cognitive and emotional processing of fictional movies. An important building block of the research program is the construct Modes of Reception. In dealing with media offers, a cinema audience, a television audience, internet users, listeners, and readers resort to individual strategies, which I call Modes of Reception. In the course of learning and socialisation processes, these strategies or modes are learned, modified and adapted through repeated application. I begin with a methodological section, in which I describe how Modes of Reception for fictional movies can be measured via a scale, which I have developed. Modes of Reception can be conceived as a trait that is essentially independent of an actual reception process but relates to previous media selection and reception experiences. This clearly distinguishes the construct from immersion, presence or narrative engagement. I then move on to a theoretical section, in which I elaborate on the most important considerations underlying the construct Modes of Reception. The possibility of differentiating recipients according to their dominant Modes of Reception combined with a diligent analysis of the movie in question and actual reception process data enables researchers to identify effects of specific stimuli. In the research program different methods are systematically combined to identify the effects of movie features in an empirical way. I exemplify the idea to conceptualize reception processes as a continuous switching between dominantly used Modes of Reception by referring to data of facial expressions gathered in a reception study using the animated short film ‘Father and Daughter’. In order to realize experiments, movies are often reduced to short stimulus sequences and/or modified for experimental variation. In my research program full-length, original movies are investigated. The greatest challenge for data analysis is to identify patterns in the sequence of cognitive appraisals, elicited emotions, and physiological reactions. In the last step, I provide results of a time pattern analysis for the reception of ‘Father and Daughter’. Suckfüll, M. (2004). Rezeptionsmodalitäten. Ein integratives Konstrukt für die Medienwirkungsforschung. München: Fischer. Suckfüll, M. (2013). Emotion Regulation by Switching between Modes of Reception. In A. P. Shimamura (Ed.), Psychocinematics: Exploring Cognition at the Movies (pp. 314-336). New York: Oxford University Press. Suckfüll, M., & Scharkow, M. (2009). Modes of Reception for Fictional Films. Communications, 34, 361-384. Suckfüll, M., Unz, D. (2015). Understanding Film Art: Moments of Impact and Patterns of Reactions. In M. S. Magnusson, J. Burgoon, & M. Casarrubea (Eds.), Discovering Hidden Temporal Patterns in Behavior and Interaction (pp. 165-181). New York: Springer.

 

DAY 2 (22.06.16)

9:15 start

Andreas Lieberoth

Having to play versus wanting to play: Methods for studying implicit motivation in gamification and citizen science.

Contrary to notions from utility oriented economic theory and classical behaviorism, much human activity ranging from play, over hobby pursuits, to bingewatching Game of Thrones, seems to be self-propelled based on concepts like interest, engagement, playfulness, sociality, curiosity and mastery. While it is quite likely that such motivations evolved in the service of phenomena like learning, bonding or stress management, and ensuring activity over inertia in the life trajectory, they are independent of immediate usefulness. They are, in a word, paratelic happening without an explicit outer-motivated goal. In this talk, I briefly outline the concepts of implicit and explicit motivation, and present two intertwined techniques we use to study them within the self determination theory framework: The behavioral experimental setup, and the self report scales used to disentangle different kinds of driving motivations in the lab or ind the wild (IMI, PENS, (c)PLOC). I discuss how we have used these techniques and theories to codify game play and learning motivations, from dissociating the effects of framing and mechanics in gamification, to introducing off-the-shelf computer games in math classrooms, to mapping participation motifs in citizen science problem solving on www.scienceathome.org. The talk will include our own practical learnings. Participants will have open access to most survey materials.

 

Félix Schoeller

Aesthetic chills as a measure of engagement/emotion

This presentation will report about the phenomenon of aesthetic chills (hedonic non-thermoregulatory shivering) and its possible scientific usage. Aesthetic chills seem to be a universal emotion experienced throughout cultures and non-thermoregulatory shivering also occurs in diverse non human species. In humans, it has been reported in relation to strong/involving stimuli such as works of art, works of science or in the course of various religious rituals. In this presentation, we will specifically focus on the case of chills as elicited by narrative films. In a series of both qualitative and quantitative experiments conducted in Europe in the course of the year 2015, we devised an experimental protocol that allows for a controlled manipulation of aesthetic chills in the laboratory. Our results suggest that aesthetic chills are inhibited by exposing the subject to an incoherent prime prior to the chill-eliciting stimulation and that a meaningful prime makes the aesthetic experience more pleasurable than a neutral or an incoherent one. Further experimentation revealed that chills induced by narrative structures seem to have a significant calming effect, while at the same time eliciting strong engagement with the story and empathy for the characters involved. An effort will be made to relate these observations to the general mechanics of film techniques and the transparency they gain from building on basic perceptual and emotional processes, transparency that will finally modulate absorption, engagement and immersion. The talk will end with an open discussion regarding if and how this measure can also be applied for research pertaining to other media and real world experiences.

Schoeller, F (2015) Knowledge, curiosity, and aesthetic chills. Frontiers in Psychology. 6:1546. doi: 10.3389/fpsyg.2015.01546.

Schoeller, F., Perlovsky, L. (2015) Great Expectations—Narratives and the Elicitation of Aesthetic Chills. Psychology, 6, 2098-2102. doi: 10.4236/psych.2015.616205. 

 

Mette Steenberg 

 

Engagement in reading

I would like to engage you all in a “shared reading” session. We’ll read a short story and a poem together under my guidance. On the basis of this practice demonstration I’d like to discuss reading engagement in two respects. 1. “Shared reading” as a form of practice that facilitates engagement: What kind of engagement and how does it differ from other forms of reading engagement. 2. “Shared reading” as a methodology for eliciting subjective accounts of qualitative dimensions of reading engagement.

Kuiken, D., Miall, D. S., and Sikora, S. (2004). Forms of self-implication in literary reading. Poetics Today, 25(2), 171-203.

Petitmengin, C.,(2006)”Describing one´s subjective experience in the second person: An interview method for the science of consciousness”, Phenom Cogn Sci 5: 229-269.

Sikora, S., Kuiken, D., and Miall, D. S. (2011). Expressive reading: A phenomenological study of readers' experience of Coleridge's The rime of the ancient mariner. Psychology of Aesthetics, Creativity, and the Arts, 5(3),

Steenberg, Mette, Pernille Bräuner & Sebastian Wallot: (2014) Text Technology: “Building Subjective and Shared Experience in Reading” Journal of Cognition and Culture 14 (4) 357–372

Steenberg, Mette (2016): Literary reading as a technology of the mind: An exploratory study on social forms of reading. In McKechnie, Lynne (E.F.) and Kjell Ivar Skjerdingstad, eds. Plotting the Reading Experience: Theory/Practice/Politics. Wilfrid Laurier University Press. Looking forward to meeting you all!

 

John Mason

Engagement in the classroom

My talk will derive much of its content from thoughts about the practical business of teaching. In particular the teaching of language. My purpose will be to draw on experience and thinking about teaching to explore how engagement/immersion do or do not come about, how they are or are not maintained and what factors seem to encourage or discourage entry through the doors of engagement and immersion in what follows. The teaching of a foreign language involves transmitting an invitation to engage in a foreign world – one that, for an identity-conscious teenager, will seem threatening. My interest is in the ways in which ontological features of human response such as anxiety, sympathy or herd instinct positively or negatively affect engagement and retention of engagement. Attempts to engage (the verb can now be both transitive and intransitive!) others in an experience that they have not selected, that has profound implications for their identity, that invites failure and loss of face, that takes place in artificial and often uncongenial environments would seem to be doomed to failure. What strategies does the poor teacher have at his/her disposal? I, too, will be looking at motivational factors like intrinsic/extrinsic motivation but more especially at the notion of ‘future selves’ – how (young) people create imagined versions of their future identity that then act as guides for behaviour. This will be set against recent thinking (e.g. Thomas Ziehe) about post-modern children and the proliferation of choice and individual determination, and against some thoughts about the notion of ‘togetherness’ in the post-modern classroom. I am hoping to be able to provide a demonstration of some aspects of my talk.

A few references: Brodersen, Hansen & Ziehe, Oplevelse, Fordybelse og Virkelyst, Reitzels Forlag, 2015

Dörnyei & Ushioda, Motivation, Pearson, 2011

Jackson, Carolyn, Lads and Ladettes in School, Open University Press, 2006

Oyserman & James, Possible selves: from content to process. In Markman, Klein, Suhr (eds) The Handbook of Imagination and Mental Stimulation, NY: Psychology Press.

Ziehe, Thomas, Lernkultur, jugendlicher Mentalitätswandel und die Relativierung der Eigenwelten (unpublished paper)

 

Some general details about the bootcamp format:

The event is part of a series that we call "IMC bootcamps". The IMC bootcamps are designed to bring together PhD students, established researchers of different fields and practitioners interested in scientific approaches (this time for example from CAVI, Kolding Design School and LEGO), to discuss and be informed about various approaches to everlasting hot topics in Cognitive Sciences. Each workshop is one or two days long and open to a small crowd of people (30-40). It is an opportunity to present latest results, but more importantly, thought of as a space to discuss different approaches (e.g. theoretical enquiries, observations, various behavioural data, physiological measures, imaging methodologies) to look at the same phenomena with regard to questions, such as: - What are the premises of these methods? What do they actually assess? Which assumptions are made when used to look at the phenomenon of interest? What conceptual implications does that have? How do they inform each other or can they be combined, considering all this, without problems? - How do we apply these questions to research involving interacting individuals (n>1 in the lab), and what happens then? To allow this, the event is structured as follows: - The morning starts with an introductory lecture, giving a short and more historical overview and presenting general outstanding questions. - Then, 2-3 methodological lectures, each presenting one approach - Finally, a informal discussion with all participants about questions that have emerged and challenges for future research. Time for talks and discussions is usually almost 50/50. See info on previous bootcamps here interactingminds.au.dk/search/local/;