Archives For author

The V&A Museum has an exciting exhibit currently open in the Sackler Centre. ‘Digital Dragons’ makes two Chinese paintings ‘come alive’ through digital projection and a whole body interface supported by Kinect devices. Stepping on the floor influences the projection of the painting: depending on where you step, the landscape painting gains colour and human activity and in the other painting, dragons begin to chase coloured pearls.

WBI V and A

At MODE we’ve been observing children as they interact with the exhibit. In a series of naturalistic observations, we videoed 1-4 year olds interacting with the exhibit with their parents. In the next phase of research, we observed 7-8 year olds from the same school interacting with the exhibit, and then interviewed them afterwards to see how they had made sense of the experience.

Our questions in conducting these observations have been broad. We’re interested in how children make sense of their movements in relation to the projected visual activity; how the involvement of the whole body in this experience might influence the children’s interpretations of the paintings involved in the exhibit; and how social interactions are configured and coordinated in this type of interactive museum space.

In-depth analysis will begin soon. Our preliminary thoughts and ideas have centred on where the students focused their attention. For the first cycle of the interaction with the paintings, many of the children seemed to focus exclusively on the colour and movement they saw on the floor. Over the course of their interaction however, they became increasingly aware that their movements were having an impact on the projected image and began to make sense of exactly what this relationship was. This might suggest that exhibits that make use of two separately located sites of activity (here the floor and the wall) may require sustained interaction if students are to understand how activity in these spaces is connected.

Advertisements

By Anna Xambó, Carey Jewitt, and Sara Price

Scanning a shoe at the Fashion Digital Studio, London College of Fashion (UAL).

Scanning a shoe at the Fashion Digital Studio, London College of Fashion (UAL).

MIDAS involves collaborators from different disciplines and backgrounds connected to Digital Arts and/or Social Sciences.  It addresses a pressing problem for contemporary research: how to synthesise approaches from the Arts and Social Sciences to develop innovative methods of research on digital technology and embodiment. It addresses this problem in the context of digital technology and the issues the digital raises for interacting with the body and the different methodological demands this places on disciplines where the body is a primary object of study, particularly in the Arts and the Social Sciences. The key objectives of MIDAS are to:

1) Describe the concepts, practices and processes used to research embodiment in digitally assisted arts (performance, fashion, design), and social sciences (medical simulation, technologies for education, online games);

2) Identify points of methodological connection and synergy across this multidisciplinary terrain;

3) Experiment how to integrate and exploit the methodological synergies and approaches to evaluate their applicability to embodiment research across the Digital Arts and Social Sciences;

4) Design training and capacity-building resources to support methodological innovation across the boundaries of the Digital Arts and Social Sciences.

MIDAS is currently investigating methods used to research notions of embodiment in different disciplinary contexts, through an exploration of six ethnographic case studies. Each site is a hub of methodological innovation, engaging in research on the body/physical interaction, and advanced digital technologies (e.g. body scanners; motion capture; or virtual environments). We are particularly interested in understanding what and how: methods are used at an institutional and individual level; body/physical interaction and digital technology are used; and methods are used for methodological innovation.

Through ethnographic observation, we try to get a sense of the methods and practices used in each site. This is complemented by informal conversations, and literature review packages suggested by each site. We are focusing on understanding how the different sites look into embodiment. As each site thinks differently about embodiment, we listen to the way they talk and observe their practices focusing on body, technology, and methods in order to get a sense of the assumptions and principles used in each site when thinking about the body. We use field-notes, photographs, and video recordings to document these ideas and routine methodological practices. This data will be analyzed to understand the different ‘methods world’ of each site. We plan to conduct a series of workshops in 2014 with the attendance of experts to explore themes, perspectives, experiences, and contribute to the development of future methods.

The MIDAS blog aims at providing a shared space for researchers and practitioners connected to the project to express their voices about research methods on digital technology and embodiment, and also finding out methodological synergies between Digital Arts and Social Sciences via documenting, reflecting, and sharing practices, processes, and ideas about digital technology and embodiment.

Project website: http://MIDAS.ioe.ac.uk

You can follow us on Twitter: @MIDAS_LKL

I found jotting down 10 things I know about phenomenology a helpful exercise for consolidating and summarizing knowledge of a central approach in embodiment research. Over the last few weeks, I have repeatedly come across the term ‘Somaesthetics’ in relation to embodiment and have done some preliminary reading on the term. In an effort to make sense of the fledgling understanding I’ve gained, I decided to try the ’10 things I know’ approach again. Without looking at books, notes or papers, I scribbled down the following:

1. Somaesthetics is an emerging discipline primarily proposed by Richard Shusterman, who argues for it to sit within Philosophy, and more specifically the philosophical sub-discipline of Aesthetics.

2. Somaesthetics as a discipline foregrounds the role of bodily experience in aesthetic appreciation. Shusterman argues that it will offer a more substantial and systematic framework for research on the body.

3. Somaesthetics comprises three branches: Analytic somaesthetics; Pragmatic somaesthetics; Practical somaesthetics

Somaesthetics Diagram

4. Analytic somaesthetics comprises interpretive studies of bodily practice. These occur on a theoretical level and would include research linking the body and bodily practices with the organization of society and the individual psyche. Shusterman argues that the work of Foucault, Bourdieu and Merleau-Ponty would all constitute analytic somaesthetics.

5. Pragmatic somaesthetics comprises bodies of thought that suggest ways of training or harnessing bodily experience. Practices such as Tai Chi and the Alexander Technique are accompanied by ideas and theories of the body and these would belong to this branch of pragmatic somaesthetics.

6. Practical somaesthetics consists of the practice itself – the Tai Chi, or the yoga, or the Alexander Technique. It is not entirely clear which bodily practices Shusterman chooses to accept within practical somaesthetics. He seems to place less value on athletics or football, and much more on solitary practices that reference bodily understanding more explicitly.

7. In foregrounding the body in aesthetic experience, Shusterman argues that a distinction needs to be drawn between representational foregrounding and experiential foregrounding. In the former, the body is seen and treated as an external object. In the latter, the body is a fundamental part or vehicle of lived experience.

8. Critiques of somaesthetics include the linking of the discipline to normative approaches to the body. Somaesthetics places a value on the physical cultivation of the body and this is reminiscent of doctrines that treat the body as an object to be molded into a form that conforms to the social ideal.

9. Shusterman’s response to this critique rests on the distinction he makes between representational and experiential foregrounding of the body. Thus, he argues that cultivating the body as it is experienced (rather than as it is represented) does not relate to social norms, but rather to the improvement of individual perception and action.

10. Shusterman advocates that there is a role for somaesthetics in education since it enables us to ‘feel better’ in both senses of the phrase. Cultivating bodily practices heightens our sensations and perceptions of the world around us; they also encourage us to be more in control and more caring of the self.

The combination of theoretical and practical approaches to the body certainly makes the discipline of somaesthetics an interesting prospect, but I am left with questions about the possibility and worth of trying to merge these perspectives. Shusterman’s framework makes such a sharp and convincing distinction between analytic, pragmatic and practical somaesthetics, that it isn’t clear how it draws these together at all. How are Foucault’s theories of the body related to the theory or practice of yoga? How can somaesthetics help us to access this relationship? Is this a relationship that should exist given how different the objectives related to each project are? Certainly, I believe that theorists of the body can learn much from bodily practices and practitioners who work with the body, but what is the benefit of binding them together in a single discipline?

Shusterman, R. (2008) Body consciousness: A philosophy of mindfulness and somaesthetics. Cambridge: Cambridge University Press.

Shusterman, R. (2006) Thinking through the body, educating for the humanities: A plea for somaesthetics. The journal of aesthetic education, 40(1), 1-21.

By Mona Sakr

An overview Loke and Robertson’s ‘Making Strange’ methodology and a discussion of its relevance to embodiment

In Loke and Robertson’s (2008) ‘making strange’ methodology, the way we perceive and feel the moving body is actively unsettled in order to find new perspectives on the body and bodily interaction. The practice of ‘making strange’ builds on the work of dancer Maxine Sheets-Johnstone who explored the phenomenological consequences when individuals disrupted their familiar or habitual movements. In ‘making strange’, our assumptions and habitual understandings about the body are unsettled through explicit inquiries into movement. Loke and Robertson argue that de-familiarising the movements of the body can open up new spaces in the design of artefacts and technologies. It can help us to invent or devise new types of movement; it enables an investigation into the experience of movement; and it encourages us to find new methods for re-enacting or recording movement.

 So how does ‘making strange’ work? Loke and Robertson suggest a range of techniques that can facilitate innovative thinking about the body and movement:

  • Scoring – a simple score might consist of three movements e.g. walking, standing and crouching, but by improvising in terms of the speed, duration, order, focus, or use of space, these movements can be radically altered.
  • Falling – unorthodox movements, like falling, can help us to re-feel and re-negotiate our relationship with the moving body.
  • Somatics – certain practices, like Qi Gong, can help us to become more aware of our felt sensations and to prioritise these over the external perception of movement.
  • Finding Pathways – what part of the body do you lead with when you move, and what would happen if this were changed? Try walking left shoulder-first, or leading with your elbow when you lift your arm.
  • Imagery – the language we use to talk about the body and movement can help us to enact movements differently. Think of the ‘light as a feather, stiff as a board’ game that you may have played when you were younger. Chanting these words as a group while lifting one of your peers changes the experience.

For dancers, these techniques are a way of accessing, inventing and designing new forms of movement. For researchers of the body though, we also need methods for capturing responses to these movement techniques. And as designers, we need to link these responses to, or ‘write’ them into, systems that depend on movement.

Some ideas for recording responses might include:

  • Video to capture the movement
  • Transcriptions of the movement through visual plots e.g. plotting the motion of particular points on the body
  • Photography to capture the moments that define the essential points of the movement e.g. when we raise our hands, the essence of the movement is in the fingers raised towards the sky
  • In situ comments on the phenomenological consequences of the movement
  • Oral or written reflective commentaries

‘Making strange’ is an exciting starting point in the design of embodied interaction. Beyond the techniques it suggests though, we need robust methods for capturing the outcomes of practising these techniques, along with guidelines for implementing them in the design of systems that depend on whole-body interaction. For me, the next step in understanding the ‘making strange’ methodology will be to try the techniques suggested by Loke and Robertson and to capture my responses in the ways I’ve suggested.  Will the outcomes be useful in thinking about and designing for whole-body interaction?

Loke, L. & Robertson, T. (2008) Inventing and devising movement in the design of movement-based interactive systems. OZCHI’08 (81-88).

By Mona Sakr

Some thoughts about the role of gaze in students’ historical inquiry a digitally augmented local site of interest

I’m currently analyzing data from a project looking at students’ exploration of a digitally augmented local site of interest. The students used iPads to engage with the WW2 history of the local common and the experiences of people in the area during the war. One of our research focuses is how being in situ can facilitate students’ inquiry about the past. How do students use both the physical and the digital environments to support their learning and interactions?

In coming to a multimodal analysis of students’ embodied experiences of time and place, I suspected that movement would be important to students’ inquiry. I had hypothesized that movement would enable them to make links between different areas on the common. However, what I hadn’t  previously thought much about was the way that gaze would also be an essential tool in enabling inquiry about the past.

Research on gaze has tended to position it as either an indicator of attention (as in psychological research) or as a key instrument in social interaction (as in sociology and conversation analysis). But in our project on students’ inquiry on the Common, gaze acted as a thinking tool. In particular, the movement of their gaze back and forth between the digital environment of the iPad and the physical environment of the local common enabled them to engage simultaneously with the past and the present, comparing these points in time.

Gaze as a thinking tool 1

Gaze as a thinking tool 2

In an illustrative clip (picture above), two students are discussing how they think they would have felt if they had had to live in a deep shelter under the common during WW2. They talk about what they would have missed and constantly their gaze moves between the image of the shelter on the iPad screen and the physical environment that surrounds them – an environment that they describe as ‘free’. Gaze enables them to regularly re-engage with the present day environment so that they can work through abstract ideas or associations they have about the space.

To see more about this project, watch our video about embodied experiences of Clapham Common and students’ historical inquiry.

By Mona Sakr

Getting to grips with the philosophical approach of phenomenology

When you work with/in embodiment, you come across so many concepts that feel almost impossible to get a true grip on. Beautiful ideas, like phenomenology, roam free and can be intimidating as a result. In my role at MODE, I am currently contributing towards a glossary of terms that are used in embodiment research. It’s a fantastic way of working out what’s really important when you’re first engaging with a new concept or approach. After writing the first draft of the glossary entry for ‘Phenomenology’, I decided to test myself. Without looking at a single book, paper or webpage, what could I say that I knew about phenomenology? The result was the following:

10 things I know about phenomenology

  1. Phenomenology is the study of lived experience.
  2. The founder of phenomenology was the German philosopher Husserl, who lived and wrote at the turn of the twentieth century.
  3. Heidegger, Husserl’s student, was fundamental in developing the field of existential phenomenology.
  4. In 1927, Heidegger wrote Being and Time, a central text in phenomenology.
  5. In Being and Time Heidegger made the distinction between tools that are ‘present-at-hand’ and ‘ready-to-hand’. Tools that are ‘ready-to-hand’ can be used without thinking, without awareness. Although familiarity usually leads to tools becoming ‘ready-to-hand’, our awareness of them may be drawn back to them if they suddenly stop working (Heidegger uses the example of the pen that breaks while you are writing).
  6. In 1945, Merleau-Ponty wrote a book called Phenomenology of Perception. It is another central text in phenomenology.
  7. In Phenomenology of Perception, Merleau-Ponty presents the body as the hub of all meaning-making. Physical touch represents a ‘chiasm’ (a crossing-over) between subjectivity and objectivity. Bodies are both capable of touching and are tangible.
  8. Dreyfus (1992) built on Heidegger’s ideas towards technology use in his critiques of artificial intelligence. He has argued that AI algorithms and devices only make sense through the implicit and tacit knowledge that a user brings to them. Studying interactions with technology are therefore as much about studying the user as they are about studying the object.
  9. How do you collect data as a phenomenologist? Phenomenologists typically access lived experience through interviews and participant reflections (either written or oral). An interview approach called Interpretive Phenomenological Analysis (IPA) is one example of a methodological framework that uses phenomenology as its starting point.
  10. Phenomenological enquiries are likely to foreground the lived experience through four ‘lifeworld existentials’ (Veletsianos and Miller, 2008): body, time, space, relations with others.

At the end of this exercise, the questions I’m left asking are:

What are the glaring gaps in my breakdown of phenomenology?

Is this a positive exercise or does it lead to the reduction of ideas?

What concepts/approaches in embodiment would I struggle to do this exercise for?

By Mona Sakr

A discussion of the term ‘intercorporeality’ and its relevance to embodiment in digital environments.

 The term ‘intercorporeality’ simultaneously foregrounds the social nature of the body and the bodily nature of social relationships. As a concept, it emphasizes the role of social interactions in the construction and behaviours of the body: ‘the experience of being embodied is never a private affair, but is always mediated by our continual interactions with other human and nonhuman bodies’ (Weiss, 1999, p. 5). At the same time, it suggests that our existence in relation to others – our intersubjectivity – is something tangible and bodily (Csordas, 2008).

 Intercorporeality is a relevant concept for understanding embodied experiences in digital environments because as Kim (2001) suggests, digital environments open up new opportunities for intercorporeal practices. Through bodies, we can share and extend our ‘bodily experiences’ (Merleau-Ponty, 1962).

Consider the following examples:

  • Through a webcam, I can see into locations that go beyond those that immediately surround me in physical space,
  • Through social media, I can extend my grip on others and touch the lives of others though they are not physically close to me (see Springgay, 2005),
  • Through my phone, I can audio record others’ voices and hear again the past and the interactions it comprised.

The words ‘see’, ‘touch’ and ‘hear’ demonstrate the extent to which social interactions are bodily. The examples above suggest that digital environments can impact on the body’s perceptions and sensations and this will, in turn, affect the way we interact with others – our intercorporeal practices.

Csordas, T. J. (2008) Intersubjectivity and intercorporeality. Subjectivity, 22(1), 110-121.

Kim, J. (2001) Phenomenology of digital-being. Human studies, 24(1-2), 87-111.

Merleau-Ponty, M. (1962) Phenomenology of Perception. London: Routledge & Kegan Paul.

Springgay, S. (2005) Thinking through bodies: Bodied encounters and the process of meaning making in an e-mail generated art project. Studies in Art Education, 47 (1), 34-50.

Weiss, G. (1999) Body images: Embodiment as intercorporeality. New York: Routledge.

By Mona Sakr

Exploring the usefulness of the ‘body-thing dialogue’ metaphor for understanding embodied interaction in digital environments

In 2006, Larssen, Robertson & Edwards presented the paper ‘How it feels, not just how it looks: When bodies interact with technology’ at the Australian Computer-Human Interaction conference (OZCHI). In this paper, they suggest that  the embodied nature of interactions with technology can be accessed by thinking about each interaction as a body-thing dialogue. The body-thing dialogue is the bodily interaction that occurs between an individual and an artifact. The body-thing dialogue happens through the mode of movement and makes possible the potential for action that forms the basis of the interaction. Through this metaphor, Larssen et al. hoped to shift the focus towards the bodily nature of interactions with technologies – the extent to which these experiences are physically felt.

I would argue that the body-thing dialogue is a useful metaphor in some ways and a misleading one in other ways.

It is useful because it draws attention to the distinct and temporal nature of each embodied interaction:

  • Every dialogue we engage in is different. Similarly, in human-computer interaction each interaction with technology unfolds in a specific way and in a particular context. Flow diagrams based on a non-existent ‘typical’ user do not help us to access the nature of embodied interaction.
  • Every dialogue unfolds over time and can radically change from moment to moment. Similarly, each embodied interaction takes place over time and is historical – each aspect of the interaction happens in relation to the aspects that have preceded it.

But the metaphor of the body-thing dialogue is also a misleading way to think about embodied interactions in digital environments:

  • Movement is a different mode to speech, with different opportunities and constraints. Is it right to apply the notion of ‘dialogue’ in the context of movement?
  • Can we apply the term ‘dialogue’ to make sense of the way movements unfold between ‘a body’ and ‘a thing’? Certainly, the movements of the body and the movements of an object are not equivalents in the way that the speech of two human participants is.
  • Larssen et al. suggest that the body-thing dialogue is a useful way of looking at ‘how we use our proprioceptive sense and motor skills when incorporating a tool in our bodily space so that it becomes an extension of our bodies’ (p. 2). The notion of dialogue takes us away, however, from the concept of incorporation. In a dialogue, there is self and other – the object responds to us, rather than becoming an extension of us.

So what alternative metaphors or conceptual tools enable us to think about embodied interaction in digital environments? I have yet to come across a theory that helps to frame interactions with artefacts so that the focus is on the body and felt experience, but does not trip into the pitfalls outlined above. We need to conceptualise interactions as physical couplings without using metaphors that draw on other modes of communication.

Larssen, A. T., Robertson, T., & Edwards, J. (2006, November). How it feels, not just how it looks: when bodies interact with technology. In Proceedings of the 18th Australia conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments (pp. 329-332). ACM.

An exploration of the term ‘affect’ and what it offers to embodiment research.

I am exploring the term ‘affect’ and what it might offer to my understanding of embodied experience. In this post, I consider how affect is talked about in research and think about the best way to use the concept in my own research on embodiment and digital environments.

Hudlicka (2003) presents affect as an essential aspect of embodied interaction with digital environments. Hudlicka uses the term ‘affect’ interchangeably with ‘emotion’. Thus, the need for computers to detect user affect is equivalent to their recognition of user emotion (whether the user is happy, sad, angry etc.). Similarly, Johnson and Wiles (2003) talk about the ‘positive affect’ associated with playing computer games and equate this with happiness, engagement or ‘flow’ (Csikszentmihalyi, 1992).

But Shouse (2005) argues that the terms ‘affect’, ‘emotion’ and ‘feeling’ must be understood and presented as distinct from one another. He suggests that while feeling and emotion are subject to recognition and labeling on the part of those who experience them, affect is a more simple individual response: ‘a prepersonal intensity corresponding to the passage from one experiential state of the body to another’. Affect is a one-dimensional spike in the intensity of experience.

If we use this definition, what distinguishes affect from arousal? Is Shouse simply arguing for the importance of measuring the physiological arousal of users? My understanding is that affect is more than arousal because it implies the existence of an external cause. An individual might experience an increase in arousal as a result of a fleeting image that passes through their mind, but affect is a consequence of being affected by another body, by something external. So, focusing on affect involves looking at not just the affective response but also at the stimulus that has caused it: the music that has moved us, or the image that has led to a pre-conscious experiential shift.

In the context of my research on embodiment and digital environments, I am interested in affect because pre-conscious responses to the external world are a vital part of user experience. But looking at the work of Shouse has convinced me that the pre-conscious nature of affect means that it cannot be measured by asking users to label their feelings or by recording visible emotions that I observe. On the other hand, measures of physiological arousal can offer some insight into affect but do not tell the whole story. To ‘get at’ affect requires looking at indicators of the intensity of experience (including physiological arousal, facial expression, vocalisations and other bodily cues) and linking these to the environment that surrounds the user.

References 

Csikszentmihalyi, M. (1992) Flow: The Psychology of Happiness. London: Random House.

Hudlicka, E. (2003) To feel or not to feel: The role of affect in human–computer interaction. International Journal of Human-Computer Studies, 59(1), 1-32.

Jonhson, D. M. & Wiles, J. (2003) Effective Affective User Interface Design in Games. Ergonomics, 46 (13/14), 1332 – 1345.

Shouse, E. (2005) Feeling, emotion, affect. M/C Journal, 8 (6), 26. Accessed online 24.08.2013: http://www.journal.media-culture.org.au/0512/03-shouse.php

An update on a current research project looking at body and space in the context of mobile technologies

By Mona Sakr

Mobile technologies have particular features that are likely to change individuals’ embodied experiences of places. Here at MODE, we’re conducting a research project to look at the influence of mobile technologies on young students’ navigation and exploration of a local site of interest and its cultural history.

We have designed a digital environment on the iPad for 10-11 year olds that allows them to explore the WW2 history of Clapham Common. The modified Evernote app, which you can see in the figure below, encourages individuals to move around the common while accessing visual, written and audio information about the history of the common. Users are also encouraged to upload their own photos and audio recordings about their experience of the common.

App environment for exploring Clapham Common

App environment for exploring Clapham Common

Our research questions probe the possibility that particular features of mobile technologies change the way individuals experience a place. In particular, we are interested in how these features change what the body is doing (the embodied experience of the place) and how this in turn affects the overall experience.

A mad flurry of data collection has happened over the past  fortnight. Working with 60 students at a primary school in Clapham, we collected a range of video data on their experiences of the common while using the mobile digital environment on the iPad. This data included researcher-generated video, headcam video and bodycam video. The video was supplemented with GPS trackings, and the photos and audio recordings that the students created while exploring the Common in pairs.

Researcher-generated video

Researcher-generated video

Headcam video

Headcam video

Before and after the experience, the students recorded their thoughts and feelings about the common via a series of classroom activities, including map-making and recounting the route they had taken on a mapped floor.

Over the next few weeks, we’ll begin to get to grips with all the data we’ve collected. As well as engaging with the research questions about embodied experiences, we are trying to determine how different forms of digital data can enable us to engage with embodiment and embodied experiences of place on an empirical level.