Archives for posts with tag: digital environments

The V&A Museum has an exciting exhibit currently open in the Sackler Centre. ‘Digital Dragons’ makes two Chinese paintings ‘come alive’ through digital projection and a whole body interface supported by Kinect devices. Stepping on the floor influences the projection of the painting: depending on where you step, the landscape painting gains colour and human activity and in the other painting, dragons begin to chase coloured pearls.

WBI V and A

At MODE we’ve been observing children as they interact with the exhibit. In a series of naturalistic observations, we videoed 1-4 year olds interacting with the exhibit with their parents. In the next phase of research, we observed 7-8 year olds from the same school interacting with the exhibit, and then interviewed them afterwards to see how they had made sense of the experience.

Our questions in conducting these observations have been broad. We’re interested in how children make sense of their movements in relation to the projected visual activity; how the involvement of the whole body in this experience might influence the children’s interpretations of the paintings involved in the exhibit; and how social interactions are configured and coordinated in this type of interactive museum space.

In-depth analysis will begin soon. Our preliminary thoughts and ideas have centred on where the students focused their attention. For the first cycle of the interaction with the paintings, many of the children seemed to focus exclusively on the colour and movement they saw on the floor. Over the course of their interaction however, they became increasingly aware that their movements were having an impact on the projected image and began to make sense of exactly what this relationship was. This might suggest that exhibits that make use of two separately located sites of activity (here the floor and the wall) may require sustained interaction if students are to understand how activity in these spaces is connected.

By Mona Sakr

Some thoughts about the role of gaze in students’ historical inquiry a digitally augmented local site of interest

I’m currently analyzing data from a project looking at students’ exploration of a digitally augmented local site of interest. The students used iPads to engage with the WW2 history of the local common and the experiences of people in the area during the war. One of our research focuses is how being in situ can facilitate students’ inquiry about the past. How do students use both the physical and the digital environments to support their learning and interactions?

In coming to a multimodal analysis of students’ embodied experiences of time and place, I suspected that movement would be important to students’ inquiry. I had hypothesized that movement would enable them to make links between different areas on the common. However, what I hadn’t  previously thought much about was the way that gaze would also be an essential tool in enabling inquiry about the past.

Research on gaze has tended to position it as either an indicator of attention (as in psychological research) or as a key instrument in social interaction (as in sociology and conversation analysis). But in our project on students’ inquiry on the Common, gaze acted as a thinking tool. In particular, the movement of their gaze back and forth between the digital environment of the iPad and the physical environment of the local common enabled them to engage simultaneously with the past and the present, comparing these points in time.

Gaze as a thinking tool 1

Gaze as a thinking tool 2

In an illustrative clip (picture above), two students are discussing how they think they would have felt if they had had to live in a deep shelter under the common during WW2. They talk about what they would have missed and constantly their gaze moves between the image of the shelter on the iPad screen and the physical environment that surrounds them – an environment that they describe as ‘free’. Gaze enables them to regularly re-engage with the present day environment so that they can work through abstract ideas or associations they have about the space.

To see more about this project, watch our video about embodied experiences of Clapham Common and students’ historical inquiry.

By Mona Sakr

A discussion of the term ‘intercorporeality’ and its relevance to embodiment in digital environments.

 The term ‘intercorporeality’ simultaneously foregrounds the social nature of the body and the bodily nature of social relationships. As a concept, it emphasizes the role of social interactions in the construction and behaviours of the body: ‘the experience of being embodied is never a private affair, but is always mediated by our continual interactions with other human and nonhuman bodies’ (Weiss, 1999, p. 5). At the same time, it suggests that our existence in relation to others – our intersubjectivity – is something tangible and bodily (Csordas, 2008).

 Intercorporeality is a relevant concept for understanding embodied experiences in digital environments because as Kim (2001) suggests, digital environments open up new opportunities for intercorporeal practices. Through bodies, we can share and extend our ‘bodily experiences’ (Merleau-Ponty, 1962).

Consider the following examples:

  • Through a webcam, I can see into locations that go beyond those that immediately surround me in physical space,
  • Through social media, I can extend my grip on others and touch the lives of others though they are not physically close to me (see Springgay, 2005),
  • Through my phone, I can audio record others’ voices and hear again the past and the interactions it comprised.

The words ‘see’, ‘touch’ and ‘hear’ demonstrate the extent to which social interactions are bodily. The examples above suggest that digital environments can impact on the body’s perceptions and sensations and this will, in turn, affect the way we interact with others – our intercorporeal practices.

Csordas, T. J. (2008) Intersubjectivity and intercorporeality. Subjectivity, 22(1), 110-121.

Kim, J. (2001) Phenomenology of digital-being. Human studies, 24(1-2), 87-111.

Merleau-Ponty, M. (1962) Phenomenology of Perception. London: Routledge & Kegan Paul.

Springgay, S. (2005) Thinking through bodies: Bodied encounters and the process of meaning making in an e-mail generated art project. Studies in Art Education, 47 (1), 34-50.

Weiss, G. (1999) Body images: Embodiment as intercorporeality. New York: Routledge.

By Mona Sakr

Exploring the usefulness of the ‘body-thing dialogue’ metaphor for understanding embodied interaction in digital environments

In 2006, Larssen, Robertson & Edwards presented the paper ‘How it feels, not just how it looks: When bodies interact with technology’ at the Australian Computer-Human Interaction conference (OZCHI). In this paper, they suggest that  the embodied nature of interactions with technology can be accessed by thinking about each interaction as a body-thing dialogue. The body-thing dialogue is the bodily interaction that occurs between an individual and an artifact. The body-thing dialogue happens through the mode of movement and makes possible the potential for action that forms the basis of the interaction. Through this metaphor, Larssen et al. hoped to shift the focus towards the bodily nature of interactions with technologies – the extent to which these experiences are physically felt.

I would argue that the body-thing dialogue is a useful metaphor in some ways and a misleading one in other ways.

It is useful because it draws attention to the distinct and temporal nature of each embodied interaction:

  • Every dialogue we engage in is different. Similarly, in human-computer interaction each interaction with technology unfolds in a specific way and in a particular context. Flow diagrams based on a non-existent ‘typical’ user do not help us to access the nature of embodied interaction.
  • Every dialogue unfolds over time and can radically change from moment to moment. Similarly, each embodied interaction takes place over time and is historical – each aspect of the interaction happens in relation to the aspects that have preceded it.

But the metaphor of the body-thing dialogue is also a misleading way to think about embodied interactions in digital environments:

  • Movement is a different mode to speech, with different opportunities and constraints. Is it right to apply the notion of ‘dialogue’ in the context of movement?
  • Can we apply the term ‘dialogue’ to make sense of the way movements unfold between ‘a body’ and ‘a thing’? Certainly, the movements of the body and the movements of an object are not equivalents in the way that the speech of two human participants is.
  • Larssen et al. suggest that the body-thing dialogue is a useful way of looking at ‘how we use our proprioceptive sense and motor skills when incorporating a tool in our bodily space so that it becomes an extension of our bodies’ (p. 2). The notion of dialogue takes us away, however, from the concept of incorporation. In a dialogue, there is self and other – the object responds to us, rather than becoming an extension of us.

So what alternative metaphors or conceptual tools enable us to think about embodied interaction in digital environments? I have yet to come across a theory that helps to frame interactions with artefacts so that the focus is on the body and felt experience, but does not trip into the pitfalls outlined above. We need to conceptualise interactions as physical couplings without using metaphors that draw on other modes of communication.

Larssen, A. T., Robertson, T., & Edwards, J. (2006, November). How it feels, not just how it looks: when bodies interact with technology. In Proceedings of the 18th Australia conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments (pp. 329-332). ACM.

An exploration of the term ‘affect’ and what it offers to embodiment research.

I am exploring the term ‘affect’ and what it might offer to my understanding of embodied experience. In this post, I consider how affect is talked about in research and think about the best way to use the concept in my own research on embodiment and digital environments.

Hudlicka (2003) presents affect as an essential aspect of embodied interaction with digital environments. Hudlicka uses the term ‘affect’ interchangeably with ‘emotion’. Thus, the need for computers to detect user affect is equivalent to their recognition of user emotion (whether the user is happy, sad, angry etc.). Similarly, Johnson and Wiles (2003) talk about the ‘positive affect’ associated with playing computer games and equate this with happiness, engagement or ‘flow’ (Csikszentmihalyi, 1992).

But Shouse (2005) argues that the terms ‘affect’, ‘emotion’ and ‘feeling’ must be understood and presented as distinct from one another. He suggests that while feeling and emotion are subject to recognition and labeling on the part of those who experience them, affect is a more simple individual response: ‘a prepersonal intensity corresponding to the passage from one experiential state of the body to another’. Affect is a one-dimensional spike in the intensity of experience.

If we use this definition, what distinguishes affect from arousal? Is Shouse simply arguing for the importance of measuring the physiological arousal of users? My understanding is that affect is more than arousal because it implies the existence of an external cause. An individual might experience an increase in arousal as a result of a fleeting image that passes through their mind, but affect is a consequence of being affected by another body, by something external. So, focusing on affect involves looking at not just the affective response but also at the stimulus that has caused it: the music that has moved us, or the image that has led to a pre-conscious experiential shift.

In the context of my research on embodiment and digital environments, I am interested in affect because pre-conscious responses to the external world are a vital part of user experience. But looking at the work of Shouse has convinced me that the pre-conscious nature of affect means that it cannot be measured by asking users to label their feelings or by recording visible emotions that I observe. On the other hand, measures of physiological arousal can offer some insight into affect but do not tell the whole story. To ‘get at’ affect requires looking at indicators of the intensity of experience (including physiological arousal, facial expression, vocalisations and other bodily cues) and linking these to the environment that surrounds the user.

References 

Csikszentmihalyi, M. (1992) Flow: The Psychology of Happiness. London: Random House.

Hudlicka, E. (2003) To feel or not to feel: The role of affect in human–computer interaction. International Journal of Human-Computer Studies, 59(1), 1-32.

Jonhson, D. M. & Wiles, J. (2003) Effective Affective User Interface Design in Games. Ergonomics, 46 (13/14), 1332 – 1345.

Shouse, E. (2005) Feeling, emotion, affect. M/C Journal, 8 (6), 26. Accessed online 24.08.2013: http://www.journal.media-culture.org.au/0512/03-shouse.php

An introduction to Biocca’s theory of ‘the phenomenal body’ and questions about its usefulness in emerging digital environments. 

In digital environments that involve an on-screen avatar, there appear to be two bodies to consider: the corporeal body through which we control the digital environment, and the digital body that appears to act upon the digital environment. Biocca (1997) has conceptualised the duality of the body across digital and non-digital environments as a constant tension or struggle to influence the phenomenal body, which can be thought of as our embodied presence in the world.

Berit

When we interact with digital environments, the phenomenal body arises as a result of experiences in both digital and non-digital domains via both the digital and the corporeal body. At times we may become so engrossed in a digital environments that our corporeal bodies are forgotten or rather, they become less important to the phenomenal body. Alternatively, if the digital body is unconvincing as a set of semiotic resources (e.g. as a result of lack of customizability), the phenomenal body will be more related to corporeal experience and less related to digital manifestations of the body.

What about in digital environments where there is no avatar? In tangible interfaces, the corporeal body influences both the physical and the digital world. The corporeal body controls (and is seen to be in control of) physical and digital representations. So in cases like this, is the corporeal body also the digital body, or has the distinction become an unnecessary or unhelpful one to make?

Biocca, F. (1997, August). The cyborg’s dilemma: Embodiment in virtual environments. Retrieved online 07.06.2013: http://jcmc.indiana.edu/vol3/issue2/biocca2.html