Archives for posts with tag: multimodality

By Anna Xambó, Carey Jewitt, and Sara Price

Scanning a shoe at the Fashion Digital Studio, London College of Fashion (UAL).

Scanning a shoe at the Fashion Digital Studio, London College of Fashion (UAL).

MIDAS involves collaborators from different disciplines and backgrounds connected to Digital Arts and/or Social Sciences.  It addresses a pressing problem for contemporary research: how to synthesise approaches from the Arts and Social Sciences to develop innovative methods of research on digital technology and embodiment. It addresses this problem in the context of digital technology and the issues the digital raises for interacting with the body and the different methodological demands this places on disciplines where the body is a primary object of study, particularly in the Arts and the Social Sciences. The key objectives of MIDAS are to:

1) Describe the concepts, practices and processes used to research embodiment in digitally assisted arts (performance, fashion, design), and social sciences (medical simulation, technologies for education, online games);

2) Identify points of methodological connection and synergy across this multidisciplinary terrain;

3) Experiment how to integrate and exploit the methodological synergies and approaches to evaluate their applicability to embodiment research across the Digital Arts and Social Sciences;

4) Design training and capacity-building resources to support methodological innovation across the boundaries of the Digital Arts and Social Sciences.

MIDAS is currently investigating methods used to research notions of embodiment in different disciplinary contexts, through an exploration of six ethnographic case studies. Each site is a hub of methodological innovation, engaging in research on the body/physical interaction, and advanced digital technologies (e.g. body scanners; motion capture; or virtual environments). We are particularly interested in understanding what and how: methods are used at an institutional and individual level; body/physical interaction and digital technology are used; and methods are used for methodological innovation.

Through ethnographic observation, we try to get a sense of the methods and practices used in each site. This is complemented by informal conversations, and literature review packages suggested by each site. We are focusing on understanding how the different sites look into embodiment. As each site thinks differently about embodiment, we listen to the way they talk and observe their practices focusing on body, technology, and methods in order to get a sense of the assumptions and principles used in each site when thinking about the body. We use field-notes, photographs, and video recordings to document these ideas and routine methodological practices. This data will be analyzed to understand the different ‘methods world’ of each site. We plan to conduct a series of workshops in 2014 with the attendance of experts to explore themes, perspectives, experiences, and contribute to the development of future methods.

The MIDAS blog aims at providing a shared space for researchers and practitioners connected to the project to express their voices about research methods on digital technology and embodiment, and also finding out methodological synergies between Digital Arts and Social Sciences via documenting, reflecting, and sharing practices, processes, and ideas about digital technology and embodiment.

Project website: http://MIDAS.ioe.ac.uk

You can follow us on Twitter: @MIDAS_LKL

By Mona Sakr

Some thoughts about the role of gaze in students’ historical inquiry a digitally augmented local site of interest

I’m currently analyzing data from a project looking at students’ exploration of a digitally augmented local site of interest. The students used iPads to engage with the WW2 history of the local common and the experiences of people in the area during the war. One of our research focuses is how being in situ can facilitate students’ inquiry about the past. How do students use both the physical and the digital environments to support their learning and interactions?

In coming to a multimodal analysis of students’ embodied experiences of time and place, I suspected that movement would be important to students’ inquiry. I had hypothesized that movement would enable them to make links between different areas on the common. However, what I hadn’t  previously thought much about was the way that gaze would also be an essential tool in enabling inquiry about the past.

Research on gaze has tended to position it as either an indicator of attention (as in psychological research) or as a key instrument in social interaction (as in sociology and conversation analysis). But in our project on students’ inquiry on the Common, gaze acted as a thinking tool. In particular, the movement of their gaze back and forth between the digital environment of the iPad and the physical environment of the local common enabled them to engage simultaneously with the past and the present, comparing these points in time.

Gaze as a thinking tool 1

Gaze as a thinking tool 2

In an illustrative clip (picture above), two students are discussing how they think they would have felt if they had had to live in a deep shelter under the common during WW2. They talk about what they would have missed and constantly their gaze moves between the image of the shelter on the iPad screen and the physical environment that surrounds them – an environment that they describe as ‘free’. Gaze enables them to regularly re-engage with the present day environment so that they can work through abstract ideas or associations they have about the space.

To see more about this project, watch our video about embodied experiences of Clapham Common and students’ historical inquiry.

An update on a current research project looking at body and space in the context of mobile technologies

By Mona Sakr

Mobile technologies have particular features that are likely to change individuals’ embodied experiences of places. Here at MODE, we’re conducting a research project to look at the influence of mobile technologies on young students’ navigation and exploration of a local site of interest and its cultural history.

We have designed a digital environment on the iPad for 10-11 year olds that allows them to explore the WW2 history of Clapham Common. The modified Evernote app, which you can see in the figure below, encourages individuals to move around the common while accessing visual, written and audio information about the history of the common. Users are also encouraged to upload their own photos and audio recordings about their experience of the common.

App environment for exploring Clapham Common

App environment for exploring Clapham Common

Our research questions probe the possibility that particular features of mobile technologies change the way individuals experience a place. In particular, we are interested in how these features change what the body is doing (the embodied experience of the place) and how this in turn affects the overall experience.

A mad flurry of data collection has happened over the past  fortnight. Working with 60 students at a primary school in Clapham, we collected a range of video data on their experiences of the common while using the mobile digital environment on the iPad. This data included researcher-generated video, headcam video and bodycam video. The video was supplemented with GPS trackings, and the photos and audio recordings that the students created while exploring the Common in pairs.

Researcher-generated video

Researcher-generated video

Headcam video

Headcam video

Before and after the experience, the students recorded their thoughts and feelings about the common via a series of classroom activities, including map-making and recounting the route they had taken on a mapped floor.

Over the next few weeks, we’ll begin to get to grips with all the data we’ve collected. As well as engaging with the research questions about embodied experiences, we are trying to determine how different forms of digital data can enable us to engage with embodiment and embodied experiences of place on an empirical level.  

  

By Mona Sakr

In imagined bodies, I talked about the way we make sense of digital communication and suggested that engaging with a blog depends on imagining the embodied actions of the blogger.

One way to think about this is through Sigrid Norris’s concept of frozen action, which she uses in order to frame the communicative work of disembodied modes. Disembodied modes are those that don’t belong to a human body e.g. writing or the organisation of furniture in this room. Norris argues that disembodied modes are understood through frozen action rather than real-time action. Frozen action is action that has been performed previously. For example, a magazine on a table is understood in relation to the actions of (1) purchasing the magazine and (2) placing the magazine on the table, although neither of those actions are currently taking place.

“These actions are frozen in the material objects themselves and are therefore evident.” (Norris, 2004, p. 14).

In this framework, disembodied systems of representation rely on deductions about embodied action. In digital environments, the perception of what others do will be based on assumptions about the embodied work underlying these events. Empirical research is needed to answer questions that arise as a result of this framework:

  • What kinds of deductions do we make about embodied work on the basis of digital communication?
  • To what extent are these deductions correct i.e. do they match the embodied work that did actually enable the digital communication?
  • What results from the way we imagine embodied work underlying digital communication e.g. does it influence the way we think about the person we are in communication with?

 

Norris, S. (2004) Analyzing Multimodal Interaction: A methodological framework. New York: Routledge.