Alex Wolfe | Project 2 | Death Stare
Some Quick Exposition
So for those of you who are unaware, Marina Abramovic is a extremely well renowned performance artist who had a retrospective in the MoMa about two years ago. For the duration of the exhibit, Abramovic performed “The Artist is Present”. After wandering through her life’s work, a visitor could sit across from Marina and behold the artist for as long as he or she could stand it. In return, Marina presents you with the world’s most perfected deadpan
The MoMa kept track of every person who sat across from Marina in the form of a headshot and a small note of how long that sat for. These are all available on Flickr, for your scraping pleasure.
Just to test the face morphing algorithm, I manually defined some basic control points on a small subset of this data. Corresponding points should be on relatively the same area of the face, so consistency is key. The more points you define, the better the final average, but the process is pretty laborious so I settled for around 40.
Taking these points, I determined a delaunay a triangulation of each face, and also a triangulation for the “average” face. In order to make a clean composite, I morphed each face to the average face by using an affine transformation on each of the triangles. Since my super useful point label sketch had glasses, I forgot to point define the eyebrows, so the eyes look a bit distorted here, but with a better point cloud
So the next step is to take an average of all of the points given, and then morph each face to that average, by interpolating the pixels inside each of the corresponding triangles. The small set I was working with while testing this algorithm was decidedly female, so the two girls below don’t suffer too much distortion, but the man in the middle gets very squished.
When you overlay all these morphed images on top of each other and blend, you get something like this
(sample set of 8 images unweighted)
sample set 8, weighted by time
sample set 25, weighted
Scraping Points with Face OSC
[vimeo http://vimeo.com/36338138 w=500&h=400]
Next I batched processed the 844 images and threw them into FaceOSC in order to automatically pick out control points. The output files averaged around 237 points, so it was far superior to my 20.
Unfortunately, in exchange for bulk, a bit of accuracy was lost. Though most face meshes generated looked like they could approximately fit the face, very rarely were they perfectly and accurately lined up in the way required for facial averaging. Most look something like the one above.
However the nice thing about the algorithm is that if you average enough points together, you get pretty close to the mean, regardless of extraneous data.
Average Face points from 250 faces
Face to Average
Comments from the pads:
Nice process shots in the blog post – baby hipsters
it’s nice that the pictures are all evenly lit and composed; weighting the persistence of the faces works well as a way to see who stayed longer; need more passes for the tracker to build a better match
how come y’all don’t have much to type this time around?
^^harder on a more abstract visualization to come up with concrete comments
Averaging as a strategy tends to remove the things that make a face unique, and emphasize obvious stuff like: 2-eyes, a nose and a mouth. Consider caricature instead: move AWAY from the the average face.
http://www.flickr.com/photos/twm_news/sets/72157628699736505/detail/
Great dataset but I agree that averaging might not have been the way to go. Too bad you ran into technical difficulties with faceosc
whats the way to go without averaging?
^maybe difference comparison? in what way?
I like the image of the averages of the points superimposed over the man’s face.
Really cool idea for a dataviz. You could also try arranging the faces in small multiples by the similarity of the faces. I think morphing them is really nice though. Its too bad you didn’t have the algorithms perfected.
I assume you’re familiar with http://marinaabramovicmademecry.tumblr.com/<----this is awesome<--whoa<---love the photos!!!! Some results came out to be like a real portrait of one person. I like to see more patterns of average images. Not sure what’s being visualized here, exactly. Maybe a clearer problem/goal statement would have helped focus things THis is beautiful!!!! The idea is really great, trying to consolidate/integrate all of the different portraits into a single space, but the morphs don’t express very much with an initial glance. i think the character angle could be a good new direction to pull out a new perspective on the data Very awesome dataset! I think that you have a seed to a larger project with the potential for a powerful outcome... There is some evidence that Di Vinci composited the face of Mona Lisa, he would dig this, and be pissed that it could be done iteratively in minutes vs/ years. Was there a comparison to Abramovic's face? I wonder if gesture becomes sympathetic, meaning individuals would bend to her will or subconciously mimic her??? i wish i were yellow +1 Dig the face averaging technique. Too bad the Face Tracker didn't really work! I wonder what other opportunities are there in morphing the faces. Perhaps incorporating Marina somehow would result some interesting directions... lolz baby hipster i like how you decided to weight it by the time they sat in the chair it was interesting to see how the faces seemed to become more femine as they were averaged nice style to intro her for the people not in the know. I see that you have points around the whole head (hairline) instead of just the face. Did you try with the face only? I have a feeling that including the head size unecessarily distored the faces, assuming the faces (and not the whole head) was the focus. It seems like time is an important quality of the dataset (in addition to expression), did you consider making some kind of animation? Maybe an accelration of a day in the gallery where faces morph to transition? Might not work since you only have static images of the faces, but it would give us more time to focus on expressions. Anyway, I think the time gets lost in the averaging, so maybe look for some other dimension to represent it in. i wonder if the overlay resulted in issues with later images having a stronger impact than earlier ones. The point scrapping animation was really cool. I think morphing would what have been pretty cool. awesome idea to take time into account Its really impressive the first mask that worked was one of your own face recognition algorithm. Not that you need to flaunt it but it was glossed over as to the logic or process of creating this CV. Just because I've spent the last few days focusing on clustering and calculating similarities between things, it occurs to me you could use the facial geometry to sort and cluster all of the faces according to their similiarty. I'm not sure if that results in anything as interesting as the baby hipster, but it's another way to push the data around. Also, that last image looks like a rendering of Kevin Kline in a video game whose clipping is failing a little. ^^ that would be cool, to be able to sort and filter through. or explore a spectrum of faces, or even dive through the layers of faces. or click on attributes and see which faces they most likely came from It may also be interesting since you have gathered the points and you can get the mean shape to run a principal component analysis so you can get variations that do not exist in the data set..or to generate variations from a set of images and then test if they exist in some other set ^^ This might be cool - eigenfaces on the geometery instead of the pixel values. Would probably help with charactures as well nice visuals. the inclusion of the artist’s face really draws me in. I like that you are using our blog post instead of a powerpoint. It seems more suitable. Very interesting idea to weight the portraits by time. Maybe since the averaging didn’t produce such an interesting effect it would have been worth trying out other effects. Like try to produce an effect to age the face or something, make it droopy based on how long they sat there? have you considered sending these face averages to the artist? I like that animation as you pour through the data. I like that you showed all the stages you went through with the project. The last set of face distortions were interesting to see the distortion of uncommon features melded altogether. Maybe do some looking into body language and its meanings. I was noticing many people either had their head tilted, mouth slightly open or eyes narrowed, you might be able to make some further conclusions based on that information? It’s interesting to see the different iterations you had and how you experimented with the data. I like that you show us obstacles of your work and how that influenced your next steps. ** Agreed. I think the project is cool, however not quite sure what kind of questions it asks. Or how would a user who is not you would explore this dataset. How about incorporating some interactivity? Where a user can play with some parameters and see how do they affect the visuals? It might look neat also if instead of layering the faces it replaced specific facial features of people, or if the faces were merged together to create a band of faces, like a motion blur I am curious about the bad data too