ARTISTS TALKING

# 26 [10 August 2009]

Michael from Fabrica has taken the camera away, to try and strip away the infra-red filter. This would be lovely if it worked. I bought a huge set of infra-red lights that I hadn't budgeted for, and so far have not seen a noticable difference. The camera sees the lights, but not to the intensity I would have hoped for. We think the infra-red filter may still be blocking a lot. So today, it will be interesting if we get a ruined camera or a unblocked camera. Either way, these are the things we have to work through this week.

I dreamt about work all night last night and jumped out of bed anxious - I need to still look at a lot more video tracks. I need to reflect a bit more about what we are understanding about the video that is there. We need to think about ways of displaying pixy that best suits the video content. At the moment, with its wood backing, its not easy to move things around. I am hoping today we can get a bit more sit and look time.

I need to edit in the last footage I have shot. Get the timings and pacings right. For some of my shoot now, they are about two hours long - which leaves for an incredibly diverse database to grap from. However - these don't work so well on Pixy - but I still want to play with scales, maybe it we just concentrate on the eyes, for example.

Each time I do a render, it takes about 4 hours for the small screen - about 13 hours for a bigger file. The file sizes are crazy.

# 25 [10 August 2009]

Its been a busy day. Trying to set up the cameras, getting the right footage shown - it seems the screen likes to show men (they sort of sway which suit the screen, where as women seem to nod, which doesn't suit the screen). This screen likes movement - but movement of a horizontal nature. This isolates me to two male tracks (I have shot about 35 portraits now). Lately I have just been working with what is around me - shooting with one light on a black background. The pixy display doesn't like the shadows this causes. So, there is not a great database to choose from.

We had the work open at Lighthouse today,  from 3-5pm. We had a few people through. Discussed the work. It was interesting to see everyone playing with the technology. Things worked - things didn't - people seemed to enjoy it though - it will be interesting to see the feedback of the evaluation that starts on Wednesday.

Jamie Wyld spent a lot of time in front of the camera. Interestingly the camera interprets him as sad, and everyone else was happy. I think in the end Jamie and the computer both got happy but it took a while.

The pixy likes faces, semi -close up with sideways movement. How do you direct for that? ;). Who knows - but we have footage to work with. It will be nice to see the second screen up and running. See more group interaction or contagion.

Jane and Karl were in again today - its been great having them about to discuss ideas with. Jane has been posting to the blog as well. 

Michael from Fabrica gallery came in - we discussed the work -= how to move ahead. It will be a busy week of testing, further building and getting a balance that suits Natacha and Michael and myself.

We had the work crash three times today. Not a good start. It seems to be something with the mind reading technology and the Pixy screen. I have sent it on to Jeff to hve a look at. 

Jeff has send back the secondary screen option - so we can run two screens from the one computer. I will install it tomorrow. I haven't heard from Gordon, but i am hoping we will have two screens to look at tomorrow.

Had a discussion with Natacha and Michael - about what they want out of this collaboration - about how they see things extending the screen. 

Camera placement is still an issue. Its like you should walk around with it. And maybe this is something that could work, but it can't... We need the lighting to follow you around as well. In my search for more naturalistic and fluid interaction I seem to have reached a few problems.

# 24 [10 August 2009]

JANE MCGRATH

Today I’m back at The Lighthouse and it was really interesting to see the Pixy up and running. It is quite startling to see how much the human mind can read and interpret from a limited number of pixels.  This makes me think a great deal about inference and suggestion.  I am planning to design, build and programme  a series of ‘bio digital ‘chairs, and the power of suggestion , contamination and transference is key. I am now encouraged to explore the notion that less can certainly be more or at least 'as much as'...

Some faces worked better, some glided across the screen with 'intense' detail and others seemed to sit back and get lost. A mans back could be read in 3- 6 pixels!

The notion of spaces inbetween and inbetween spaces is cropping up – in the physical sense of this work, ie the spaces between the pixels the spaces between the batons, the spaces between the viewer and the screen.  The experience of walking into the pixy screen  is also interesting. For me the experience of  being inside the screen was ‘live’ when I was face onto the pixels (with my back to the audience. ) When I turn forward I see only the back of the batons and I am disappointed  -  I am left wanting to see the pixels all around  - Now I have this strange urge to run in a forest of pixels,  knocking into them and seeing faces and figures every where I turn. May be a I have a weird deep seated  urge to be inside  a machine??

We had an interesting chat about the space between the action on screen and the reaction of the user and how the software reacts  to the users face.  About the moments when we are left waiting.  I love the idea of working with that (un)comfortable waiting, those odd frozen moments. Tina explained the timings of the  video reactions and how these are programmed and what work is ongoing  at MIT re further development.  Its fascinating.

I want to use my practical work to dig deeper into those ‘moments’ – those spaces of ‘pure potentiality’ – moments when  there is a temporal suspension.  When magic happens –or maybe doesn’t.    When we must wait and see.

After a very interesting lecture by Jonathan Gilhooly  at Brighton Uni we were talking about film (the old fashioned stuff)  and the spaces between the frames, we wondered when watching a film actually how many blank spaces of a film strip we actually see (even if the brain did not register them – they still exist .)   I’m sure it’s a very high percentage of blank spaces that we filter out … I would like to explore those spaces – who knows what films they could hide.. also what happens in the digital realm, the spaces between the pixels, the Pixy is a great place to explore and I’m really excited to see it up and running.

# 23 [8 August 2009]

spent the day editing video of the shoots this week. Trying to take my voice out of the sound track, and then impprting it into after effects to get the size, the levels. It taken ages, but i have done two shoots now - and tomorrow will begin an the edit points - lists of in and out points.

Beautiful day in Brighton. Sun was shining- the beach was packed. My son loved it. We took him to the park in the beach and he played in the paddle pool.

Michael and Natacha came to dinner. Matt cooked Moroccan Lamb. We didn't talk about work at all. That was lovely. it was great just to hang out and not think about it.

# 22 [7 August 2009]

Over the Lighthouse Residency, an aim is to work on two more sculptural versions of Chameleon Prototype 08. We are also working with Kim Byers and Nadia Berthouze from the HCI lab at UCL to test how people are responding 'empathically' to the different screens. Gordon Brand was at Lighthouse for the day yesterday - and we tested different shapes to project upon. We were in the secondary studio at Lighthouse - the same studio that I have been using to shoot new digital portraits. Gordon thinks that by Tuesday afternoon he might be able to have three secondary prototypes worked out. By the end of yesterday, we discussed having a screen with three layers. A back transparent layer with a slight concave shape. That feeds into a transluscent white flat surface. This meets a concave white surface that meets th flat surface in the middle. 

I need to send Gordon some video to project on the experiments. I need to do this tomorrow. I am hoping that Gordon can deliver the screens on Tuesday afternoon.

I am having trouble with the cameras of the face tech reading technology... My firewire cameras don't work with it. uSB cameras do work - but they don't like dark light. I have bought a huge range of infrared lights but so far, other than putting them directly in front of the face - I can't see a difference. I was hoping for more of a 'flood' of infra-red light. So far. not working. at all.

Gordon Brand has bought some pencil cam to test, and we are hoping to embed these in the screen. The placement of the cameras is difficult. Ultimately, the cameras want to be right in front of the face which ofcourse, doesn't work conceptually at all. Both distracting, can't see image, and more and more. don't know how we will ever work through this. We need a zoom camera. More resolution. manual zoom, USB camera that copes well in dark light. I have gone through about seven cameras now.  I am not sure where or if the type of camera I want exists. 

# 21 [7 August 2009]

I talked to Jeff Mann today in regards to the screens we are building with Gordon Brand. Jeff is based in Berlin. He is an artist who developed a lot of work around machines, society, technology - more sculptural machines. He is also a great maestro on max msp/jitter and he has built  a pretty dependable patch that creates the video engine  for the Chameleon project - from prototype 07 onwards.  Today, I discussed whether we can have the one mac-mini running two screens. This is mainly to cater towards the tech budgets limits of the project. For example, I'd like to have 12 projections, but not need to have 12 computers. I might be able to run two copies of the Max patch, with low-resolution video. If that works, is it possible to run two separate copies of the face reader, each with its own camera? Or, can you make one face reader able to accept connections from two clients, and just send the same emotion data to each client? We could have two screens for one machine? It would save a lot of time (installing preparing machines), money, freight, instal.

I am shooting High Definition for all footage - and an aim has been to show the project HD. We have managed to get a mac mini triggering real time HD footage (at this stage 1440 x 900) with the emotion face reading technology (the emotion reading technology is quite processor heavy, but  Rana El Kaliouby from MIT Media Lab and Youssef kashef from the American University in Cairo have managed to really lower the CPU, meaning we can still trigger HD footage.

... but getting hold of enough  HD video projectors isn't easy, or cheap. Few galleries have HD projectors. I have two HD monitors to test upon in the studio- but an aim of prototype 08 is to try to break beyond screens and move into more sculptural dimensions. So - can we bring the resolution down ? - between 480 x 360 or 720 x 576 pixels - will we see the break down, will this effect the way we read the image? will it be distracting?. Can the balance of the  bluring of the screen balance with the slight pixelation? I have asked Jeff to render two sizes out and see how the processing power is. We then need to get it working with the face tech. By Tuesday. ;). 

# 20 [7 August 2009]

In the afternoon we had a session discussion about the next screen. Its all running behind schedule - but now we get to see what does and doesn't work with the screen. I like the volumetric part of the screen and the image - the face feels holographic in some way - like its some sort of ghost image or something. Natacha doesn't think it works that well. I discussed ways that it could escape from its ply-wood background. we talked about hanging it, with leads. We talked about creating a 'curtain' like screen that you could walk through. I think touching and walking though the screen would be fantastic. Natacha has asked me not to call it a screen, but call it a pixy.

And I guess its this - the more sculptural elements that would be great to explore. how the work views from all angles. The pixy wants you to touch it walk through it. It would be nice to explore it - but the current iteractivity demands that you stay near to the camera that assess your facial expression - here there is a disconnect - and one that we can't work out over this residency. 

We talked about hanging it from a sort of wire, weighting it with leads, finding a rubber tubing. Creating more of a curtain. Its the time to test these things - so that seems great. It would be great to further extend - in a direction that suits natacha and michael. Micheal seems excited by the curtain idea and natacha felt it would work better for the image. I think it would be good to explore. They both say that it will be easier to install - which would definitely be great.

# 19 [7 August 2009]

We have the pixy screen working and my first thought is that is looks great. This is exciting.

Karl has arrived. We are putting a black background up. I am going to look at the ratios of the video. 

Re - look at edit points.

then placement of cameras.

# 18 [7 August 2009]

Trying to get the timings worked out for this residency has been consuming...when will things be finished by? Natacha and Michael don't really know. I am trying to push things back, but still trying to push things ahead. Now that the first screen is working we can take a breath - but everyone has been working hard. Really hard.

I didn't realise the screen ran with 240 volts of electricity running through it - if you touch it your zapped. It feels dangerous with my little son running around!

For some reason Natacha and Michael haven't bought the plastic tubes that were used when I saw the screen built in Canada. This poses a huge health and safety issue - and I don't want to show it until its worked out. Natacha and Michael agree. I hope today we can work through it, - Natacha has bought plastic sheeting to wrap around it. I imagine that this will need to cut and wrapped around the length of pixels

I was hoping the screen may have been released from its plywood backing, and you may have been able to walk through the screen - but this is not possible for this screen. However, the screen has much more resolution to it.  I have created a lot of video to test on it - have done one shoot for the screen  and then spent some time in aftereffects playing with different scales - a video more focused jut on the eyes and mouth, a video focused on head and shoulder, and then a video shot from the waste up. I have chosen the portraits that talk a bit - to get a secondary channel working. 

I am now thinking it would be great to have the Pixy project run from one screen. Jeff's email mentioned the one computer runnig two cameras. This is exciting. 

So, all is good ,really. Things are running. It feels like a big step forward. We are seeing the potential of something beautiful. more seductive.

# 17 [7 August 2009]

Its been a really busy time - today is Friday, the 6th Day of the residency. I have a fantastic time yesterday with Gordon - working through ideas, playing, experimenting. Gordon is a great person to discuss ideas with and work through how to make things happen. He is informative, inventive and also passionate about what he is doing. I wish I had more time to spend at the Rapid prototyping lab. The possibilities are exciting.  The results are exciting - simple but seductive - feels perfect for the portraits, they reveal themselves and blur war - they look like they are peering in from a misty window. . We hope to move to the second sage of the rapid prototyping on Tuesday - maybe hanging the work next week and bringing the HCI guys in to test it out. Jane, Gordon and I spent quite a few hours discussing size, placement of camera, lighting, shape, projection placement, and creating a more, sculptural screen. The results felt good - like these portraits were revealing themselves and hiding themselves. It really suited the thematics of the work.

We talked about introducing touch to the screen - but how to go about it - I am at a stage where I need to simplify this project, not introduce more elements into it. But the way we were grabbing the screen - Reaching forward with two hands to touch it - in that action of intention - there was something really beautiful - I will discuss it further today with Jeff Mann who is building the video engine. I am also discussing with Jeff the option of using two screens running from one machine. It would really cut the tech costs of the project. Hopefully we can start doing tests by next week.

Jeff has written a patch so the work talks to the pixy project. He has done a great job of it. 

Michael Roy, from EE was here all night last night trying to get the pixy screen to work. We are running about two days behind schedule. Its been difficult - first was hung and then with the worng view point - to view it you needed to be in the other room. Then getting it to work properly has taken a lot of dedication from Micheal and natacha. However, now its running it looks good - we are just about to connect it to the Chameleon video engine. I hope it works OK.  Michael has been tired - he seems and a little stressed about it all, but now its running, he seems to have relaxed. He is laughing now, smiling. 

We have had a lot going on, and making sure everyone feels included in the process has been at the forefront of my mind. We are hear to experiment and play, and keeping the 'environment' happy is crucial to people feeling like they can explore and share ideas.