ARTISTS TALKING

# 21 [7 August 2009]

 

I talked to Jeff Mann today in regards to the screens we are building with Gordon Brand. Jeff is based in Berlin. He is an artist who developed a lot of work around machines, society, technology - more sculptural machines. He is also a great maestro on max msp/jitter and he has built  a pretty dependable patch that creates the video engine  for the Chameleon project - from prototype 07 onwards.  Today, I discussed whether we can have the one mac-mini running two screens. This is mainly to cater towards the tech budgets limits of the project. For example, I'd like to have 12 projections, but not need to have 12 computers. I might be able to run two copies of the Max patch, with low-resolution video. If that works, is it possible to run two separate copies of the face reader, each with its own camera? Or, can you make one face reader able to accept connections from two clients, and just send the same emotion data to each client? We could have two screens for one machine? It would save a lot of time (installing preparing machines), money, freight, instal.

I am shooting High Definition for all footage - and an aim has been to show the project HD. We have managed to get a mac mini triggering real time HD footage (at this stage 1440 x 900) with the emotion face reading technology (the emotion reading technology is quite processor heavy, but  Rana El Kaliouby from MIT Media Lab and Youssef kashef from the American University in Cairo have managed to really lower the CPU, meaning we can still trigger HD footage.

... but getting hold of enough  HD video projectors isn't easy, or cheap. Few galleries have HD projectors. I have two HD monitors to test upon in the studio- but an aim of prototype 08 is to try to break beyond screens and move into more sculptural dimensions. So - can we bring the resolution down ? - between 480 x 360 or 720 x 576 pixels - will we see the break down, will this effect the way we read the image? will it be distracting?. Can the balance of the  bluring of the screen balance with the slight pixelation? I have asked Jeff to render two sizes out and see how the processing power is. We then need to get it working with the face tech. By Tuesday. ;). 

 

 

# 22 [7 August 2009]

 

Over the Lighthouse Residency, an aim is to work on two more sculptural versions of Chameleon Prototype 08. We are also working with Kim Byers and Nadia Berthouze from the HCI lab at UCL to test how people are responding 'empathically' to the different screens. Gordon Brand was at Lighthouse for the day yesterday - and we tested different shapes to project upon. We were in the secondary studio at Lighthouse - the same studio that I have been using to shoot new digital portraits. Gordon thinks that by Tuesday afternoon he might be able to have three secondary prototypes worked out. By the end of yesterday, we discussed having a screen with three layers. A back transparent layer with a slight concave shape. That feeds into a transluscent white flat surface. This meets a concave white surface that meets th flat surface in the middle. 

I need to send Gordon some video to project on the experiments. I need to do this tomorrow. I am hoping that Gordon can deliver the screens on Tuesday afternoon.

 

I am having trouble with the cameras of the face tech reading technology... My firewire cameras don't work with it. uSB cameras do work - but they don't like dark light. I have bought a huge range of infrared lights but so far, other than putting them directly in front of the face - I can't see a difference. I was hoping for more of a 'flood' of infra-red light. So far. not working. at all.

Gordon Brand has bought some pencil cam to test, and we are hoping to embed these in the screen. The placement of the cameras is difficult. Ultimately, the cameras want to be right in front of the face which ofcourse, doesn't work conceptually at all. Both distracting, can't see image, and more and more. don't know how we will ever work through this. We need a zoom camera. More resolution. manual zoom, USB camera that copes well in dark light. I have gone through about seven cameras now.  I am not sure where or if the type of camera I want exists. 

 

 

# 23 [8 August 2009]

spent the day editing video of the shoots this week. Trying to take my voice out of the sound track, and then impprting it into after effects to get the size, the levels. It taken ages, but i have done two shoots now - and tomorrow will begin an the edit points - lists of in and out points.

Beautiful day in Brighton. Sun was shining- the beach was packed. My son loved it. We took him to the park in the beach and he played in the paddle pool.

Michael and Natacha came to dinner. Matt cooked Moroccan Lamb. We didn't talk about work at all. That was lovely. it was great just to hang out and not think about it.

# 24 [10 August 2009]

JANE MCGRATH

Today I’m back at The Lighthouse and it was really interesting to see the Pixy up and running. It is quite startling to see how much the human mind can read and interpret from a limited number of pixels.  This makes me think a great deal about inference and suggestion.  I am planning to design, build and programme  a series of ‘bio digital ‘chairs, and the power of suggestion , contamination and transference is key. I am now encouraged to explore the notion that less can certainly be more or at least 'as much as'...

Some faces worked better, some glided across the screen with 'intense' detail and others seemed to sit back and get lost. A mans back could be read in 3- 6 pixels!

The notion of spaces inbetween and inbetween spaces is cropping up – in the physical sense of this work, ie the spaces between the pixels the spaces between the batons, the spaces between the viewer and the screen.  The experience of walking into the pixy screen  is also interesting. For me the experience of  being inside the screen was ‘live’ when I was face onto the pixels (with my back to the audience. ) When I turn forward I see only the back of the batons and I am disappointed  -  I am left wanting to see the pixels all around  - Now I have this strange urge to run in a forest of pixels,  knocking into them and seeing faces and figures every where I turn. May be a I have a weird deep seated  urge to be inside  a machine??

We had an interesting chat about the space between the action on screen and the reaction of the user and how the software reacts  to the users face.  About the moments when we are left waiting.  I love the idea of working with that (un)comfortable waiting, those odd frozen moments. Tina explained the timings of the  video reactions and how these are programmed and what work is ongoing  at MIT re further development.  Its fascinating.

I want to use my practical work to dig deeper into those ‘moments’ – those spaces of ‘pure potentiality’ – moments when  there is a temporal suspension.  When magic happens –or maybe doesn’t.    When we must wait and see.

After a very interesting lecture by Jonathan Gilhooly  at Brighton Uni we were talking about film (the old fashioned stuff)  and the spaces between the frames, we wondered when watching a film actually how many blank spaces of a film strip we actually see (even if the brain did not register them – they still exist .)   I’m sure it’s a very high percentage of blank spaces that we filter out … I would like to explore those spaces – who knows what films they could hide.. also what happens in the digital realm, the spaces between the pixels, the Pixy is a great place to explore and I’m really excited to see it up and running.

# 25 [10 August 2009]

Its been a busy day. Trying to set up the cameras, getting the right footage shown - it seems the screen likes to show men (they sort of sway which suit the screen, where as women seem to nod, which doesn't suit the screen). This screen likes movement - but movement of a horizontal nature. This isolates me to two male tracks (I have shot about 35 portraits now). Lately I have just been working with what is around me - shooting with one light on a black background. The pixy display doesn't like the shadows this causes. So, there is not a great database to choose from.

We had the work open at Lighthouse today,  from 3-5pm. We had a few people through. Discussed the work. It was interesting to see everyone playing with the technology. Things worked - things didn't - people seemed to enjoy it though - it will be interesting to see the feedback of the evaluation that starts on Wednesday.

Jamie Wyld spent a lot of time in front of the camera. Interestingly the camera interprets him as sad, and everyone else was happy. I think in the end Jamie and the computer both got happy but it took a while.

The pixy likes faces, semi -close up with sideways movement. How do you direct for that? ;). Who knows - but we have footage to work with. It will be nice to see the second screen up and running. See more group interaction or contagion.

Jane and Karl were in again today - its been great having them about to discuss ideas with. Jane has been posting to the blog as well. 

Michael from Fabrica gallery came in - we discussed the work -= how to move ahead. It will be a busy week of testing, further building and getting a balance that suits Natacha and Michael and myself.

We had the work crash three times today. Not a good start. It seems to be something with the mind reading technology and the Pixy screen. I have sent it on to Jeff to hve a look at. 

Jeff has send back the secondary screen option - so we can run two screens from the one computer. I will install it tomorrow. I haven't heard from Gordon, but i am hoping we will have two screens to look at tomorrow.

Had a discussion with Natacha and Michael - about what they want out of this collaboration - about how they see things extending the screen. 

Camera placement is still an issue. Its like you should walk around with it. And maybe this is something that could work, but it can't... We need the lighting to follow you around as well. In my search for more naturalistic and fluid interaction I seem to have reached a few problems.

 

# 26 [10 August 2009]

Michael from Fabrica has taken the camera away, to try and strip away the infra-red filter. This would be lovely if it worked. I bought a huge set of infra-red lights that I hadn't budgeted for, and so far have not seen a noticable difference. The camera sees the lights, but not to the intensity I would have hoped for. We think the infra-red filter may still be blocking a lot. So today, it will be interesting if we get a ruined camera or a unblocked camera. Either way, these are the things we have to work through this week.

I dreamt about work all night last night and jumped out of bed anxious - I need to still look at a lot more video tracks. I need to reflect a bit more about what we are understanding about the video that is there. We need to think about ways of displaying pixy that best suits the video content. At the moment, with its wood backing, its not easy to move things around. I am hoping today we can get a bit more sit and look time.

I need to edit in the last footage I have shot. Get the timings and pacings right. For some of my shoot now, they are about two hours long - which leaves for an incredibly diverse database to grap from. However - these don't work so well on Pixy - but I still want to play with scales, maybe it we just concentrate on the eyes, for example.

Each time I do a render, it takes about 4 hours for the small screen - about 13 hours for a bigger file. The file sizes are crazy.

# 27 [11 August 2009]

For this blog, I am asking different people who have been involved to add to the blog. Karl has been invaluable over the last week, to discuss ideas, and help build the work. He is interested in the exploration of emotional contagion and interaction in the Chameleon Project. 

I have had to cut this blog into four parts, the a-n blog allows a post of 500 words at a time

 Dr Karl Broome Is currently a research fellow in the sociology department working on the project 'Supporting Shy Users in Pervasive Computing', an EPSRC funded research project on the WINES programme, undertaken by the departments of Informatics and Sociology at the University of Sussex

 

KARL BROOME: PART ONE

 

For Walter Benjamin, the term mimesis refers to the compulsion to
'become and behave like something else'. When I sit and look at the Pixy
screen I think and feel about the 'emotional contagion' that is to be
brought about through my 'interaction' with the screen in front of me.
Strictly speaking, it is not straightforwardly the screen in front of me
with which I am interacting with but more the small camera, which is
'reading' and 'responding' to my facial expressions, or rather, it is
interacting with me. But that said, sometimes it is kind of like looking
at the reflection of your own face in the ripples of water, except it is
not a reflection of your own face that you are partially seeing, but a
response to what is perceived as your emotional state, embodied in the
expression of another human being.

# 28 [11 August 2009]

 

For this blog, I am asking different people who have been involved to add to the blog. Karl has been invaluable over the last week, to discuss ideas, and help build the work. He is interested in the exploration of emotional contagion and interaction in the Chameleon Project. 

I have had to cut this blog into four parts, the a-n blog allows a post of 500 words at a time

 Dr Karl Broome Is currently a research fellow in the sociology department working on the project 'Supporting Shy Users in Pervasive Computing', an EPSRC funded research project on the WINES programme, undertaken by the departments of Informatics and Sociology at the University of Sussex

KARL BROOME PART TWO
The low-resolution Pixy screen aims to provide a sense of 'immersion'
but with minimal detail. When choosing which 'emoting' characters filmed
as part of the Chameleon project are to be shown on the Pixy screen,
details such as the profile of the shoot- is the person filmed side on
or face on- are taken into consideration as this affects to what extent
a face can be seen when interacting with Chameleon, and arguably to what
extent one can identify the emotional response of the character on the
screen. But after sitting and watching the screen for some time, I kind
of feel that emotional contagion is achieved whether or not the face is
identifiable – 'emotional' immersion is still achieved... To borrow from
Brian Massumi , what 'objectively speaking' we see when we look at the
Pixy screen is squares of green light (pixels) in vertical lines, but
what we 'feel' we see is the faces of a man or  a woman crying, or
laughing hysterically. I still have the compulsion to feel like the few
pixels on the Pixy screen, even when they no longer ‘objectively’
resemble a face. But how does one ‘feel’ like pixels on the screen if
they only appear as nothing more than movement?.. perhaps it is the
result of my prior understanding or experience of the screen when the
face was more clearly identifiable, partially the product of memory, but
I am not so sure...  I have a sense that the Pixy screen can potentially
express emotions without the need of a face being discernible in the
pixels. Am experiencing a compulsion to become and behave like something
else?

 

# 29 [11 August 2009]

 

For this blog - I am asking different people who have been involved to add to the blog. Karl has been invaluable over the last week, to discuss ideas, and help build the work. I have had to cut this blog into four parts, as with the a-n blog you are only allowed to write 500 words at a time

Dr Karl Broome Is currently a research fellow in the sociology department working on the project 'Supporting Shy Users in Pervasive Computing', an EPSRC funded research project on the WINES programme, undertaken by the departments of Informatics and Sociology at the University of Sussex

KARL BROOME: PART 3

Working within the space as the team construct and fine-tune the
'exhibit' one cannot but help occasionally quiver with the affective
energy that reconfigures what is simply a rectangular white walled
basement room into a socio-sensual space of affective intensity. The
room exhibits an intense expressivity even before the lights go off and
Chameleon, Prototype 8 is in full operation. Throughout the day we hear
the repetition of certain ‘emotive’ phrases as the Chameleon films are
playing in the background as we work– emotionality is fore grounded
within the space across both audio and visual registers... 

"Fuck..Fuck", I can hear sound of the emotional expletives being made by
a man crying as I walk around the computer monitor, this makes me feel
for lack of a better word ‘uncomfortable’. Tina explains "he has just
lost his wife, poor man" ... when I hear that the man crying on the
screen has genuinely experienced the traumatic loss of a loved one, I
instantly feel sadness in the pit of my stomach...` in retrospect the
feeling I experienced must have been a strong sense of sympathy,
empathy.. I suppose I felt his loss. Or rather, what started out as an
uncomfortable feeling was translated into a painful sense of loss when
accompanied with Tina’s account.  Initially it was just the sound of the
man's voice making me feel that way...Tina’s account intensified the
feeling, provided it with meaning, gave it shape, made it semiotically
ordered.

# 30 [11 August 2009]

For this blog - I am asking different people who have been involved to add to the blog. Karl has been invaluable over the last week, to discuss ideas, and help build the work. I have had to cut this blog into three parts, as with the a-n blog you are only allowed to write 500 words at a time

Dr Karl Broome Is currently a research fellow in the sociology department working on the project 'Supporting Shy Users in Pervasive Computing', an EPSRC funded research project on the WINES programme, undertaken by the departments of Informatics and Sociology at the University of Sussex

KARL BROOME - PART 4

Today Chameleon, Prototype 8 was open to the public.  Time was spent
altering the position of the camera used to ‘read’ the facial
expressions/emotional states of interactants, and in testing out the
overall running of the software on the Pixy screen. Before we knew it,
3pm had arrived and a group of around six young adults descended into
the space to experience what prototype 8 was all about.  Tina, Michael,
and Natacha all contributed to explaining the different aspects and
workings of prototype 8. When watching the group of visitors interact
with Prototype 8, it seemed that the experience of the emotional
contagion in this context is socially negotiated. Although the emotional
expressions of the film on the pixy screen is the outcome of how the
software responds to the ‘emotional data’ provided by the individual
face captured by the camera, the ‘meaning’ - and arguably the emotional
experience - of this interaction in this situation seems to a
considerable degree to be socially shaped. For example, if prototype 8
responds to an individual’s facial expression with anger or sadness, the
individual’s response is not necessarily angry or sad, but rather it
appeared to be significantly dependent upon how his or her companions
also responded to the sounds and images produced through the interaction
– that is, the individual’s emotional contagion appeared to be affected
by collective response. That said there were a number distractions in
the form of a few 'teething problems' that may have impacted upon
visitors experiences. Nevertheless, when reflecting upon my own
experiences and observations within the space, it would appear at first
glance that the social context in which Prototype 8 is experienced makes
a significant impact upon levels of emotional contagion and mimicry. It
will therefore be very interesting to observe over the next few days
whether or not individuals appear to experience more emotional contagion
when interacting with Prototype 8 on their own.