Page 2 of 6 :

This project blog »

Project blogs

Tina Gonsalves, Chameleon Project, Prototype 08 - Lighthouse Residency August 2009

By: Tina Gonsalves

 

The Chameleon project is built over ten prototypes (2008-2010) with a cross disciplinary group of an artist, social neuroscientist, emotion neuroscientist, affective computer scientists, technologists, human computer interaction scientists and a curator. The project investigates the scientific foundations of emotional contagion. Supported by the Wellcome Trust, Arts Council England, Australia Arts Council,  ANAT, Lighthouse,  UCL, MIT Media Lab, Solent University and SCAN.

 

click to expand/collapse 

# 46 [14 August 2009]

 

I first met Experential Electricae at the Liminal Screens residency at the banff new media institute in Canada. They had built Pixy in the constraints of a basement in banff. The problems of finding space and hanging in banff lead the screen to be more volumetric. Natacha, presented Pixy as "conceived for both it's own use and collaboration with other video artists who would create other content for the object or even live performance". This intrigued me. A lot of artist and technologists are creating 'tools' for other artists to then appropriate. Does this really work? I was both interested in the screen and also interested in this nature of collaboration. In April I  first contacted Experential Electricae about becoming involved with the chameleon Project. We then met briefly in Vienna where I was delivering a talk about the Chameleon Project.

While trying to understand where we have come to with Pixy, I have looked back on the dialogue we have had together about Chameleon and Pixy coming together. This is an extract of the first email ever sent to Natacha regarding Pixy, and how it may work with the Chameleon Project. I looked at the screen yesterday and thought about how it had developed for the Chameleon Project.

Interestingly I don't think the low resolution would affect the emotional reading of the piece. Neuroscientific research has done tests on how much information you can take from an image of a face, and still read the emotional tone of the face. Its amazing how much tonal range you can take away. 

 I proposed  to create a screen that hangs from the middle of the room. You can see four different faces - one to the north. south, east, west. Their is mind reading technology that interacts with the viewer. The face in the video responds to the emotional expression of the viewer and tries to begin a dialogue with them. If you walk through the screens (into the mind of the work), the screen will react as well. Maybe getting annoyed or something.

 A lot of what i am trying to say in this work is about the delicate nature of our inner and outer world, and how we are constantly adjusting to what is around us. I like the delicate nature of your screens- and the fact that you don't know where the sort of begin and end. They are not 'contained', much like our emotions - always a bit leaky and prone to infection. The lovely messy bits that make us human, and that technology loves to deny.

 It would be a great collaboration, and interesting for the space. It would probably require a grant to be written to develop the screen a bit more to work in four dimensions and to install it. Who is developing this work? Is it Michel or you? I remember you mentioning you were working with another artist as well.

 what do you think? I think it would be great'.

 

# 45 [14 August 2009]

Experential Electricae’s Pixy display is gorgeous in the dark. We enter a nearly black room. We get stuck looking at fluttering pixels, at a part of the room they make sense reading a face, on the other side of the room, we just see flickering pixels. It feels like a digital forest.

I love seeing people’s faces when the image reveals itself to them – and they ‘get it’. However, the contagion aspect I was hoping to explore seems to not work. When we look at Pixy, we are more ‘perplexed’ than any thing else. We are trying to work out what is going on - I was hoping the screen may reveal and more visceral bodily response. When I look at it 'i feel in my head' more than my body.

The sound track gives us more information about emotion, but there is a disconnect. I wonder with pixy if you need more ambient sound? People think Pixy is a sound spectrograph or something.  Some people sit in front of it for ages and don’t see anything. Their face reveals more confusion than anything else. Again, confusion is not something I am exploring for this project.

For Chameleon we introduced a lot more pixels for legibility of image with in Pixy. But other than that, we didn’t experiment much with the hanging of the screen. Maybe we needed four weeks and not two? A lot of building was done, leaving little time to try and work out what else we could do with Pixy to suit the Chameleon content? I know Michael loves to see the faces on Pixy, I sense natacha has enjoyed it too, but there is still a disconnect. I think we needed to shape the display to the project look at the display from all sides. Also, the current interaction of Chameleon demands that you drive it - This definitely doesn't work for Pixy - in my ultimate pixy we would walk through it, as if entering the body of emotions, and they would be fluttering around it. We see it from all angles.

On Wednesday Helen Sloan came in, as well as Matthew Miller and Micahel Maydon from Fabrica. We look at both iterations of the work. Helen walks into the room with the screens we have built with Gordon from Solent University - she loves they way they have revealed and shown faces - that you can see one side clear and the over more ambiguous. She then walks into pixy and she sees a completely different rendering of the same project. But this one is more 

I need to arrive at a decision of how to progress with these iterations.


tina Gonsalves, Chameleon Project, prototype 08.

[enlarge]
tina Gonsalves, Chameleon Project, prototype 08.

# 44 [14 August 2009]

 

I haven’t written over the last couple of days because my time has been so stretched, and my thoughts are so embedded and confused that I haven’t been able to talk about approach, nothing seems too clear right now.  Its been busy, bus, busy over this residency – and now I need time out to reflect -  more in reflecting than doing. My thoughts are scattered, trying to reflect - and the reflection is at a space that I still haven’t been able to translate into words. But now, I guess I will try. Start thinking about what has happened over the last ten days.

I have shot 5 different portraits, each a really amazing journey of attempting to read people, give and take. I have enjoyed the journey, the people have given and shared and often exposed a lot. I then have sat and analyzed and categorized that footage – more as looking at narrative and science of what an emotion is.  This process has felt harsh – as if I am fragmenting a lovely relationship, objectifying it, making it into a production.

Over the Lighthouse Residency I have been developing new ways of looking a displaying video, both with the pixy project and experiential electricae with the rapid prototyping group at Solent Universiy.

Over the last couple of days I have tried to understand pixy and how it reveals and image, and what images may break through the challenges of the pixy display. Last week I tried shooting for Pixy, concentrating on voice and movement. Much of that didn’t work at all. We worked out the screen liked closeness of the face, really cropped in – it liked a sort of swaying movement in order to capture glimpses of a face. I created many different video tracks to put through – few worked. Pixy likes great lighting and my recent video portraits have been shot in a studio with one light – I decided to be more adaptable and respond to content and not be so focused on the technicalities of lighting. In the end we were down to two men – one called Kevin and the other Simon. Not particularly long portraits – they repeat themselves pretty quickly. I have a bout 35 portraits now. Some of these are a few hours long.

 

# 43 [14 August 2009]

Dr Karl Broome Is currently a research fellow in the sociology department working on the project 'Supporting Shy Users in Pervasive Computing'. He has been at Lighthouse over the last ten days - helping, writing notes -

 “Oh, oh. Is it on? I am not being filmed am I?” The man seems to beconcerned with whether or not his face, and more importantly his facial expressions are being recorded. Our core five facial muscles work throughout the day expressing emotions; concurrently others are constantly reading our facial expressions.  Numerous people have come forward to allow Tina to film them expressing a broad range of emotions in front of the camera. As Tina has commented, this is literally an intensively ‘emotional’ experience with the process often resulting in those being filmed breaking down in tears. A very intimate and moving process for all involved, and Tina is aware of her privilege in going through these very personal journeys with her participants, who she has acknowledged have revealed very personal and precious information to her. Visitors to the Chameleon exhibit have frequently asked if it is their own face displayed on Pixy. Despite not being able to recognise whether or not it is actually their face,  discovering whether it is in fact their face seems to make a considerable difference to how they experience the ‘image’, and perhaps more importantly how they think others may experience the image. Maybe the experience of being 'read', and thus presented back through Chameleon, inaugurates a new moment in the experience of 'myself as other' (see Celia Lury’s ‘Prosthetic Culture’). In some sense, the concern exhibited by visitors maybe the thought that Chameleon has potentially taken something profoundly personal and defining of the individual: their emotional self and made it ‘public’. Where as a photo image, or a film can be taken to retain the 'visible' surface expressions of ‘self’, Chameleon through reading these expression probes even more deeper, and understood as revealing more deeper thoughts and feelings. Is there a fear that the 'mind reader' technology has the power to reveal the 'truth' of who we are – how we feel 'inside'?  Thus make publicly available our neuroses, reveal our 'inner demons', warts and all? Or maybe still, make public that which we are unaware, something like our 'unconscious' selves. In a sense, the emotional dialogue afforded by Chameleon shows “look what my emotions and feelings do to other people”. The video in front of us becomes a reverberation of ourselves, we feel 'I am responsible for this' – ‘I am driving this’. If we choose to 'interact' with Chameleon, we have little scope for strategically controlling the reception of our mediated self-presentation, and its subsequent reverberations.  According to Baudrillard, we now exist in culture where we live as if we have a video recorder in our heads:  we are always transforming ourselves in anticipation of what we might look like as an image (Lury 1998:78). Perhaps it is this heightened self-awareness, and anticipation of ourselves, or of visual emotional response attributed to our ‘selves’ being made a ‘image’ that makes people feel more comfortable in interacting with the Pixy than interacting directly with the face reading software for any length of time.

# 42 [12 August 2009]

Conner - 16 years old, spent last week work on the project

 

Saturday 1st August:

Today there was an artist talk at the Lighthouse Arts Gallery in the Digital Lounge. The talk lasted for around 1hr 20mins and consisted of the artist,Tina Gonsalves, talking about the  history of Chameleon Project and where it is today. Tina was joined by some 'Specialists', one of whom was Natacha who is from the Experimente Electricae Programme in France.

 

Monday 3rd - Wednesday 5th August:

During this time, I washelping Natacha from Experimente Electricae build her 3D 'PIXY' screen that the Chameleon Project is being shown on. This ment that I was doing all sorts of different things from glueing the pixels onto wood so that they could be hung from the ceiling and and also wiring the pixels together to be placed on the wood. I even made a pixel line near enough from scratch which allowed me to see how the pixels are put together.

 

Michael, from Experimente Electricae, was at Lighthouse on the Tuesday and Wednesday and he was the person who built the circuit board that will allow the 'PIXY SCREEN' to work. I also done an Hours worth of transcribing for Tina but only managed to get through 3 minutes of the tape. All in all, for my first experience, working with digital artist was interesting and fun as everyone was kind and I'd like to thank them all for allowing me to help out, so THANKYOU everyone, especially Gen, who was the experience host and the Head Co-ordinator at Lighthouse.

experientiae electricae, chameleon project. Photo: tina Gonsalves. Courtesy: experientiae electricae. The first shot of Pixy revealing a face. Its incredibly hard to shoot. 

[enlarge]
experientiae electricae, chameleon project. Photo: tina Gonsalves. Courtesy: experientiae electricae. The first shot of Pixy revealing a face. Its incredibly hard to shoot. 

experientiae electricae, chameleon project. Photo: Tina Gonsalves. audience testing pixy 

[enlarge]
experientiae electricae, chameleon project. Photo: Tina Gonsalves. audience testing pixy 

tina Gonsalves, Chameleon Project. Photo: Tina Gonsalves. Testing the rapid prototyping screens.

[enlarge]
tina Gonsalves, Chameleon Project. Photo: Tina Gonsalves. Testing the rapid prototyping screens.

tina Gonsalves, Chameleon Project, prototype 08. mocking up the screens from the rapid prototyping lab

[enlarge]
tina Gonsalves, Chameleon Project, prototype 08. mocking up the screens from the rapid prototyping lab

Tina Gonsalves, Chameleon Project, prototype 08. Photo: Tina Gonsalves. testing the rapid prototyping screens. There is elements here I really like that suit the group interaction of the work. Somehow there is more of an intimacy.

[enlarge]
Tina Gonsalves, Chameleon Project, prototype 08. Photo: Tina Gonsalves. testing the rapid prototyping screens. There is elements here I really like that suit the group interaction of the work. Somehow there is more of an intimacy.

# 41 [11 August 2009]

its hitting 9pm - still working. Gordon couldn't deliver the screens so I have mocked up some ones so when they come tomorrow it won't take long to install. The open exhibition period came and went pretty quickly. We have the computer running two screens. This is great, as the guy at the mac store told me that this wasn't possible - the graphics card wouldn't be able to handle it. The resolution is OK - I still haven't worked out how this is working with the camera, but anyway, a great progression.

I don't like the fact that the pixy asks you to be outside of the room to interact with it - or to find its viewing spot. At the same time, I hate the way my computer is sitting in front of the screen - people become so aware of it and then forget about the screen. Tomorrow we will fine tune it again - get the systems outside of the room - so we can still control it - but don't have everyone looking at it. This will be a good step forward.

For the current version of the Chameleon Project the work demands that you  stand infront of the mind reader for it to work - I don't think this works with Pixy - I am wondering if a small change of narrative can happen. for example - the character stays walking back and forth until you engage with it - and then the face comes clearer - this allows people to walk around the room more. I will ask jeff about it tonight.

I haven't heard from Rana, Youssef or Abdehlrahmen about the crashes, Jeff Mann who has developed the video engine has said its not on his end. Tomorrow I will test it with the old version of the face reader and see what happens. 

Kim from the UCL interaction lab arrived today. I have been trying to gather participants for the evaluation. Today we gathered another three or four. It make take longer. Thank god we didn't start it yesterday.

Michael Roy seems to have the better luminosity sorted out now. Natacha has been spending the day getting the next pixy display together. This one is going to be hung as a curtain. I was hoping it may have been hung as a curve, but this can't happen which is ashame. Anyway, the next one should be up tomorrow.

I have asked jeff to make the pixy be driven by the one computer. We need some more computers now - but it takes time to set them up and I sort of need Jeff to do that.

We are using a new camera that seems to respond better to the infra-red - but doesn't seem to latch onto your face as easy as the other one. The logitech one didn't work anymore. Michael from fabrica tried pulling out the infra-red sheild - and it didn't work. strange. 

 

 

 

 

 

tina Gonsalves, Chameleon Project, prototype 08. kevin on the new screens

[enlarge]
tina Gonsalves, Chameleon Project, prototype 08. kevin on the new screens

chameleon project, prototype 08

[enlarge]
chameleon project, prototype 08

# 40 [11 August 2009]

KARL BROOME

 

Chameleon sets out to explore the scientific foundations of emotional
contagion. Utilising the six key emotions of disgust, happiness, anger,
neutrality, sadness and surprise originally identified by Paul Ekman,
the ‘face reading’ software attempts to identify the emotional
expressions of the participant. As the individual emotes, both their
facial expression, and potentially their ‘mind’ are ‘read’. How should
we understand these emotions in the context of the ‘emotional dialogue’
that Prototype 8 affords: where do they sit on the explanatory continuum
with biological explanations on one end and social on the other? 
Sociological debates concerning emotions have been characterised with
the conceptualisations of emotions varying across a continuum with
‘organismic’ approaches on one end, ‘social-constructionist’ accounts on
the other, and ‘social-interactionist’ accounts somewhere in the middle.
At the ‘organismic’ end, we would find the likes of Charles Darwin, and
Paul Ekman, emphasising the innate, biological and ‘pre-cultural’ basis
of emotion and their expression - causes of emotion are wired in the
brain for instinct and survival. At the other end of the continuum we
find the ‘social constructionist’ accounts of emotion that have stressed
the  ‘social’ nature of human emotions, understanding the emergence of
emotions in terms of their social, cultural, and historical variability,
meaning and experience, with the biological being understood as largely
irrelevant. For Social interactionists, emotions are recognised as
having biological substrates, but socially shaped and subject to
hierarchical manipulation. In contradistinction to constructionist
accounts, Interactionists recognise the importance of biological
process, and recognise the ‘embodied nature’ of emotions.  Thinking
about ‘visitors’ experiences of interacting with the Chameleon project
provides an interesting opportunity to revisit some of these polarities
in sociological theorising.  The social ecology of the space in which
works such as Chameleon are exhibited significantly impacts upon
affective experience of the work. Mundane material, physical and spatial
elements, and their affordances in terms of movement, interaction,
proximity, distance and visibility all play their part in terms of
interaction with the work, and the various forms of social interaction
taking place.  Observing and interacting with people visiting the
exhibit today I became aware of people's reluctance to stand in front of
the face -reading camera for any extensive period.  Visitors appeared to
be much more comfortable in entering the dark room where the only source
of illumination was the relatively small amount of light produced by the
Pixy 'screen' - they appeared to be much more comfortable and interested
in watching Pixy from inside the room.  I heard various people comment
upon how they felt Pixy to be the most interesting part of the work,
although not quite understanding how it worked. I feel that at this
testing/evaluation stage it takes a bit of active engagement and time to
really experience what it is that makes Chameleon so special- at the
moment visitors seem somewhat distracted by the presence of the computer
monitor and the face reading camera. Undoubtedly, with a few more tweaks
visitors will experience a collaborative work of an exceptional intensity.

crashes

[enlarge]
crashes

Tina Gonsalves, Chameleon Project, screen grab. crashes of the chameleon project.

[enlarge]
Tina Gonsalves, Chameleon Project, screen grab. crashes of the chameleon project.

tina Gonsalves, Chameleon Project. the video engine, emotion face reader working. The video engine has been built with the help of Jeff Mann - the Face Reading technology has been developed by Rana El Kaliouby from the MIT media lab.

[enlarge]
tina Gonsalves, Chameleon Project. the video engine, emotion face reader working. The video engine has been built with the help of Jeff Mann - the Face Reading technology has been developed by Rana El Kaliouby from the MIT media lab.

tina Gonsalves, Chameleon Project. the video engine and the mind reader of the Chameleon Project working. It tracks key points of the face.

[enlarge]
tina Gonsalves, Chameleon Project. the video engine and the mind reader of the Chameleon Project working. It tracks key points of the face.

# 39 [11 August 2009]

So far - the face reader seems to be crashing. I am still to work out whether this happens only when the pixy screen is added. So far - when I simulate the face reader I get no crash. When I have the face reader running with the pixy screen we get a 'timeout'. when I run the video engine without the pixy screen - so far no crash.

There are so many different parts to this project - the face reader - the algorithms - now pixy - the video engine - its hard to know how to get to the bottom of it all. 

I am hoping Gordon comes in today with more screens...its looking unlikely so far... not a great start. However, past 1pm I will start setting up next door - Jeff sent the new video engine that runs two screens.

 

 

# 38 [11 August 2009]

In June when I was in Brighton working with Fabrica about how we might approach the autumn exhibition, I shot Alice-Gale. She wasn't available for a third shoot, which was ashame. Below Alice writes about the experience of being in front of the camera and being asked to express different emotional states.

Alice-Gale:

For me it was a real eye opener; to become aware of the unsaid emotions and feelings that have been pushed to the back of my mind after the years of new experiences and new memories. 
The project was very prominent in my mind for the two days of my involvement,and I am still currently thinking about my responses, what I could have said etc. What i found most consuming was the thoughts about my emotions or lack of emotions and what this meant about me as a person.
During the shoot,  I felt i sometimes tried to convince myself of sadness or happiness as my mind almost went blank in front of the camera (an unnatural setting to expose yourself) I then became aware that I had not felt real strong emotion for a long time , or thought I hadn't; in the attempt to recreate the emotion, the feelings flooded back.


I felt quite moved both during and after as it bought up for me feelings about friends, relatives that i had forgotten or moved on from in the business of my mind and life. Sometimes i felt regret about thoughts and feelings..I feel I've learn t about the necessity of holding back on having to vocalize feelings when it could trigger a consequential negative response in myself.
I was surprised that i found it difficult, but equally surprised how liberating it was; I am more inspired to be honest with myself about emotions from now on.

Tina Gonsalves, Chameleon Project. still from Amy Portrait, the Chameleon Project

[enlarge]
Tina Gonsalves, Chameleon Project. still from Amy Portrait, the Chameleon Project

# 37 [11 August 2009]

I shot Amy in the studio last week. It was our third shoot together. Its been great to get to know her a little bit more. Its interesting to look at the footage, as the more we got to know each other, the more at ease her face and body became. In the end, she revealed a lot of body movement, with out particularly realizing it. It looked like she could have been a dancer in another life.

 

Amy: 

I’d never been in front of a camera before so I had to ajust fairly quicky to talking to someone and thinking about things whist looking down a very close lense and being lit by a spot light. I did managed to feel the emotions I was asked to recall a little as I spoke, but being in such an unfarmilier environment it was hard to remove myself from it, athough by the second day things where somewhat easier.  How easy the emotion was to act out depended for me on how recently the incident I was talking about had occured, the event i talked about for “sad” was only days old and very fresh in my mind, but for “disgusted” and perticually for “angry” they were years old, so although they were both very strong instances of those emotions i felt more as if i was distantly remembering how i felt rather than truely recalling it.

One peculiar thing was although I produce some artwork myself which I have always wanted to be quite personal (I am currently doing an art foundation at the age of 24) I found recalling emotions when asked very hard. I do have a wealth of exeriances to draw from emotionally, but, (possibly because of having only done so under therapy conditions), I am incredebly guarded about talking and perticually about recalling such things.

Page 2 of 6 :

This project blog »

Tina Gonsalves

Gonsalves' current work investigates the intersections of art, technology and science.  She is currently working with world-leaders in psychology, neuroscience and emotion computing in order to research and produce moving image artworks mobile and wearable technolgy works respond to emotional signatures of the body. Tina Gonsalves is artist in resident at the Wellcome Department of Neuroimaging London, UK, MIT Media Lab, Cambridge, USA, Nokia Research Labs Tampere, Finland and Brighton and Sussex Medical School Brighton, UK.