Archives for posts with tag: gesture

By Mona Sakr

Some thoughts about the role of gaze in students’ historical inquiry a digitally augmented local site of interest

I’m currently analyzing data from a project looking at students’ exploration of a digitally augmented local site of interest. The students used iPads to engage with the WW2 history of the local common and the experiences of people in the area during the war. One of our research focuses is how being in situ can facilitate students’ inquiry about the past. How do students use both the physical and the digital environments to support their learning and interactions?

In coming to a multimodal analysis of students’ embodied experiences of time and place, I suspected that movement would be important to students’ inquiry. I had hypothesized that movement would enable them to make links between different areas on the common. However, what I hadn’t  previously thought much about was the way that gaze would also be an essential tool in enabling inquiry about the past.

Research on gaze has tended to position it as either an indicator of attention (as in psychological research) or as a key instrument in social interaction (as in sociology and conversation analysis). But in our project on students’ inquiry on the Common, gaze acted as a thinking tool. In particular, the movement of their gaze back and forth between the digital environment of the iPad and the physical environment of the local common enabled them to engage simultaneously with the past and the present, comparing these points in time.

Gaze as a thinking tool 1

Gaze as a thinking tool 2

In an illustrative clip (picture above), two students are discussing how they think they would have felt if they had had to live in a deep shelter under the common during WW2. They talk about what they would have missed and constantly their gaze moves between the image of the shelter on the iPad screen and the physical environment that surrounds them – an environment that they describe as ‘free’. Gaze enables them to regularly re-engage with the present day environment so that they can work through abstract ideas or associations they have about the space.

To see more about this project, watch our video about embodied experiences of Clapham Common and students’ historical inquiry.

by Victoria Grace Walden 

Speculations on a cinema of moments.

Tom Gunning (1995), in his paper An Aesthetic of Astonishment: Early Film and the (In)Credulous Spectator, defines the cinema that preceded classical narrative films “the cinema of attractions”. He describes it thus:

The aesthetic of attraction addresses the audience directly, sometimes… exaggerating this confrontation in an experience of assault. Rather than being an involvement with narrative action or empathy with character psychology, the cinema of attractions solicits a highly conscious awareness of the film image engaging the viewer’s curiosity. The spectator does not get lost in a fictional world and its drama, but remains aware of the act of looking, the excitement of curiosity and its fulfilment.

 (Gunning in Williams (ed) 1995: 121)

Gunning’s cinema of attraction is an attack on the senses; a cinema of moments which engages the spectator with the fundamental essence of cinema: the look. He is aware that the cinema of attractions did not completely disappear with the advent of classical narrative cinema. When you watch many films from the classical, new and contemporary eras of Hollywood there are evidently sequences which could be classified under a cinema of moments. These moments are perhaps not “attacks” on the senses, but certainly sensual moments which encourage the spectator to comprehend the cinematic image through the body. These images highlight what Laura U. Marks (2000) calls “haptic visuality” – images which provoke sensations of touch and movement. I argue, it is not just the “act of looking” which audiences are aware of in these moments, but the act of sensing, through all the senses: through the body. These “moments” are asides from the narrative. Through foregrounding the body they encourage the spectator to consider how the body is socialised within the film’s context: issues of gender, power and emotion become the focus rather than narrative storytelling. They could be considered corporeal monologues or asides.

Let me share three brief examples:

The Great Dictator (1940)

Ballet sequence: Slapstick is a genre conventionally characterised by “moments”: comical set pieces. In Charlie Chaplin’s commentary on Nazi Germany, the ballet sequence is a typical example of the cinema of moments. Chaplin signifies his aside by stating “leave me. I want to be alone.” This is followed by a majestic ballet routine in which he bounces a balloon which represents the globe. The dance illuminates the absurdity of man as all-powerful. In the moment in which he holds the whole world in his hand, Chaplin’s dictator loses his power. As he spins the balloon, lies on the desk, flicks the balloon with his buttocks and gazes at it romantically, his gestures feminize the body. He is beautified – an image of performance and theatre; a subject of the gaze, quite contrary to the strong Aryan worker concept of masculinity. The disappearance of the balloon emphasises his bodily gestures and signifies the end of the aside.

It Happened One Night (1934)

Hitchhiking: When Ellie Andrews and Peter are walking down a deserted road, she dictates the start of the aside when she decides to “go sit right down, over here” on a roadside fence. Peter’s carrot munching is reminiscent of Bugs Bunny’s iconic image establishing him as rough and perhaps animalistic. The gag that ensures focuses on gestures. Peter illustrates a variety of hand gestures, each more confident and stronger (and more absurd) than the one before. The certainty of his final gesture mimics the patriarchal mythology of the strong handshake and the importance for a man to perform his dominance through gesture. As Peter fails to hail a car his gestures become less controlled, more frantic and finally plain rude.

In contrast, Ellie’s first attempt is successful. Rather than bothering with firm hand gestures, she simply raises her skirt to expose her thigh. Interestingly, this sequence does not portray the female as object for the male gaze (as Laura Mulvey notes in many films of the time (1975)), rather the female controls it. Ellie gains power over Peter, the driver and the spectator by telling all of us when and where to look at her body. This is a fabulous sequence illustrating how performance of the body emphasises gender difference.

Taxi Driver (1976)

You talkin’ to me: In this scene, most famous for its dialogue, the concentration of medium and medium close-up shots of Travis emphasise it is a sequence about the body. Again, it is established as a narrative aside: Travis is alone in his apartment. Travis begins by pacing and crossing his arms, the latter motion is an attempt to establish a defensive position. Then he begins to flick his gun out rhythmically. As he does so, the gun and the body’s movements become one – a symbol of his body at its prime: his body as war machine. He later turns his back slightly to the spectator signifying he is gaining control over the fictitious conversation. Travis establishes himself as choreographer of the scenario – when he turns away, he stops talking and the interaction ends. After a pause of motionless, Travis turns once more towards the camera flicking the knife with ease and thus establishing his violent intentions. He is a threat. He dominates. He is restoring patriarchal power to his body and to his country. The sequence illustrates the crisis of masculinity imposed by the Vietnam War and Travis’ attempt to restore order.

These are just quick thoughts on a notion and not completely theorised as yet, but to think about a cinema of moments we must think through the body. Haptic moments are not confined to the avant-garde, the most mainstream narrative stories are interrupted by such moments which help us to reflect on the body and its social role.

By Mona Sakr

A few days ago, I talked about Streeck’s taxonomy of gesture in Gesturecraft. It’s now time to share the taxonomy of hand action we’ve developed at the lab in response to a study of the hands in scientific inquiry. This taxonomy relates particularly to scientific inquiry contexts (though it may be useful for looking at hand action in other forums of experience) and is based on ‘reading’ both the form and function of hand actions. It’s inspired by literature in the field and video analysis of students involved in inquiry learning about the behaviour of light.

1.     Ergotic movements

Ergotic movements are those that change the surrounding environment . Such movements may involve changing the position of an object, or attempting to change its physical properties. In the context of scientific inquiry, ergotic movements are necessary in order to facilitate observations of particular phenomena.

2.     Epistemic movements

Epistemic movements are those that enable an individual to know more about the physical properties of an object. While ergotic movements are designed to change the surrounding environment, epistemic movements enable better perception of the surrounding environment e.g. through feeling the texture of an object.

3.     Deictic gesture

Deictic gestures are used to point to or physically highlight objects or areas in the physical world. They may be used to draw attention to a representational field or a particular aspect within a field.

4.     Re-enactment gestures

While deictic gestures draw attention to particular parts of the environment, re-enactment gestures focus on descriptive processes and so have an added temporal dimension of expression. Through using re-enactment gestures processes that are otherwise too fast to be visible can be slowed down.

5.      Ideational gestures

While all of the actions described above relate to physical phenomena that are present, ideational gestures can be used to indicate content that is not present in any respect, like abstract ideas or previous experiences. In the context of scientific inquiry, students may wish to invoke previously learned knowledge in order to make sense of what is currently occurring. Gesture may be helpful in this because it constitutes a way of representing absent knowledge.

By Mona Sakr

Embodiment isn’t just about recognising the importance of the body. It highlights the need to ‘read’ bodily actions and make sense of the semiotic work that these actions are doing. In a current project, I am attempting to make sense of the work done by different hand actions in the context of scientific inquiry. Is it possible to map hand actions to stages in scientific inquiry?

‘Reading’ parts of the body requires classification systems – systems that will make sense of all of the types of movement that can be performed. Hands might be used to move objects, to ‘know’ the texture or weight of objects, to gesture at objects present, or to gesture about objects absent. Systems designed to classify these types of hand action are a starting point for making sense of embodied forms of interaction.

One such system – based very much on function rather than form – is presented by Streeck (2009) in Gesturecraft. Streeck presents 6 categories of hand action, or, as he calls them, ‘gesture ecologies’. What do you think of the distinctions he draws?

1. Making sense of the world at hand (moving and touching objects)

2. Disclosing the world within sight (drawing attention to a shared visual focus e.g. through pointing)

3. Depiction (gestures used to represent content)

4. Thinking by hand (gesture that facilitates thought e.g. grasping at the air when you are trying hard to describe something)

5. Displaying communicative action (showing or foreshadowing aspects of the communicative act)

6. Ordering and mediating transactions (regulating the input of other participants; managing your interaction in an exchange)

But there are problems with classification systems that work purely on function, just as there are problems with those that relate purely to form. If we classify movements and actions only on the basis of function, how do we go about making the classification? It becomes an activity that relies entirely on ‘reading’ the surrounding context – you don’t end up ‘reading’ the body at all. For example, in order to know whether someone is currently ‘thinking by hand’, I would need to know what’s going on in their mind.

On the other hand, systems based only on form (e.g. those that distinguish pointing from grasping) are only helpful when making sense of the body’s semiotic work if you assume that the work of the body is inflexible – that form maps neatly, one-to-one, onto function. We know that isn’t the case… we know that pointing isn’t always about establishing a shared reference point, and we know that establishing a shared reference point isn’t always done through pointing.

What you need is a system that takes both form and function into account, one that encourages the user to think about the surrounding context – the ‘multimodal ensemble of activity’ (Goodwin, 2001) – but at the same time enables some insights to be made on the basis of the body itself.

Streeck, J. (2009) Gesturecraft: The manu-facture of meaning. Amsterdam: John Benjamins Publishing Comapny. 

By Mona Sakr

Title: “Stories and embodied memories in dementia”

Speaker: Lars-Christer Hyden, Linköping University, Sweden

Today’s seminar at the London Knowledge Lab (offered by MODE in conjunction with NOVELLA) was a fascinating insight into the role of the body in the articulation of memories by individuals suffering from dementia. The seminar considered video analysis of interviews as a way of re-thinking our conceptualisations of memory and memory loss, and the importance of the body in storing, retrieving and communication memories.

Lars-Christer Hyden is a professor of Social Psychology at Linköping University in Sweden and director of the Centre for Dementia Research (CEDER). His research focuses on communication and narrative in the context of dementia. He is currently conducting studies into the experience of dementia among couples and the various semiotic tools that are used in storytelling by individuals with dementia.

Hyden’s work uses the notion of embodiment in two ways. Firstly, he draws attention to the communicative body. He argues that there is a need to explore how the body is used as a semiotic tool in the construction of narrative, particularly when language is negatively affected by dementia. Secondly, through the concept of embodied cognition, he suggests that we can begin to question traditional models of memory that highlight abstract and analytical cognition as opposed to embodied experience. Recent research has demonstrated the extent to which memories are based on sensory experience rather than abstract events.

In the seminar, we looked at a short video clip from an interview with a woman who has dementia. By focusing on gaze, gesture and language, Hyden demonstrated the extent to which language and gesture had different semiotic functions in this context. While language is analytical, gestures are syncretic i.e. they are conducted and interpreted as complete ‘wholes’ of communication. As a result, gestures were used to facilitate communication when the language necessary to communicate a particular idea wasn’t available.

Taking this idea further, Hyden argued that depending on the semiotic tools we use to share memories with others, the nature of memory changes. When we expect memories to be communicated via language, they become analytical and disembodied entities. By focusing on the role of the body in communicating memories, we are focusing on the embodied experiences that are the basis of memory formation and thereby re-thinking the construct of memory.

Hyden’s conclusions were followed by a range of observations and questions from the audience. To mention one example – an audience member drew attention to suggestions that language use itself becomes more syncretic as dementia progresses. These findings would support the importance of gesture as a tool to convey ‘wholes’ of experience, but it also suggests that within modes, different types of semiotic work can be achieved. Language is not necessarily analytical, and perhaps gesture is not necessarily syncretic.

By Mona Sakr

‘Just as others have argued that it is through our ways of seeing the world that we become viewing subjects (see Crang, 1997), it is through our bodies that we become doing subjects.’ (p. 261)

Jaworski, A. & Thurlow, C. (2011) Gesture and movement in tourist spaces. In C. Jewitt (Ed.) The Routledge Handbook of Multimodal Analysis. Abingdon: Routledge, pp. 253 – 263.