Billy Keyes – Project 4 – Video Looper

by Billy @ 5:31 am 27 March 2012

Inspiration

I first thought of this idea a year and a half ago, when I saw the following video created by Tell No One (Luke White and Remi Weekes). I believe they achieved this effect with careful compositing and I wanted to replicate it interactively and automatically. So at the risk of further highlighting the flaws in my implementation, I think sharing this best explains what I set out to accomplish.

(I recommend you check out their other videos as well)

In particular, I really like the abstract forms that are generated by freezing motion and I designed my system with the goal of generating form rather than scenes.

The plot thickens…

I thought this would be an ideal project for figuring out how to use the Kinect with openFrameworks, two things I was familiar with but had never seriously explored. But when I showed Duncan and some other friends the video above, they were immediately reminded of a game made by Double Fine called Happy Action Theater. As a part of this game, there’s a mini-game called Clone-O-Matic, which is exactly what I was planning to do but on a timer and with still images instead of video. I decided to go ahead with my project anyway.

VideoLooper

The video shows the process of recording loops (accelerated) and then the final products. Before you start recording, there’s a setup screen where you can adjust the thresholds and the angle of the Kinect. Having good threshold settings is important if you want the object to appear to float instead of appearing attached to a limb or other prop.

Technically, the software is pretty simple. The depth image from the Kinect is filtered with a median filter to reduce the noise on the edges of objects and then thresholded. The thresholded image is blurred and then used as the alpha channel for the color frame. Initially, this step was severely broken because the depth and color images were not aligned. Thankfully, the development version of ofxKinect includes built-in calibration functionality. Once I discovered this, my alignment problem disappeared. Loops are stored in memory as a sequence of frames, so the software will quickly slow down with long loops or high-resolution input.

Improvements

The biggest current problem is usability. Starting and stopping recording is awkward because you have to use the mouse or keyboard; automatic triggering or a footswitch would be better. Second, you have to look at the computer to see your position in the frame (and the video is mirrored) — a projector would help. Finally, the layers are stacked with the oldest layer on top. I suspect this is counter-intuitive and some other z-ordering scheme should be used.

The next thing to improve is the video quality. It would be great to have a higher resolution camera, but more importantly, the segmentation and looping can be improved. Like most Kinect projects, I had trouble getting clean depth data, but if this were cleaner, segmentation would be better. I’d also like to explore automatically finding loop points that minimize the visible skip when the video layer loops.

And of course, the output is only as good as the input, so I’d like to see other people give it a try.

1 Comment

  1. This is awesome. +2 I especially like when you do the cow mobile. Using the depth threshold for compositing is a wonderful idea.
    Great music, brings the video to life. +1
    Fantastic! New tools for animation and human sculpture… great great great
    I love the animation style; seems very Terry Gilliam to me. :) Nice job on the demo real, too, lots of excellent examples. I especially love the stuffed animals.
    I love the cows haning on the air. Awesome idea of work!
    It’s great how complex the video becomes from such a simple concept – I think you realized your vision of creating crazy alien creatures from basic human video segments.
    this last section of the video turnted out great, the use of editing out yourself really worked out well.I can see htis idea being used for a lot of cool stuff, awesome job! :)
    Looks really good! Loved the idea, the interaction looks fun and engaging.
    this is the kind of thing that would keep someone amused at home for hours

    The cows sold me.
    ^yes!
    I’d show the cows sooner in the documentation video.
    Really awesome use of the kinect’s abilities to create video masks. I think this would be interesting to apply to other video projects as well.
    This is really great!! Haha good music too+1
    REALLY GOOD. The piece is pretty well demo-ed. I want to see you give this to someone who has not developed it and see how creative they can get with it. Your explanation makes it sound like a very elegant interface that others could use.
    Your specificity of cutouts is sick !
    Good clean fun, excellent work, fucking awesome cows
    See Norman McLaren’s 1964 animation, “Canon”, http://www.youtube.com/watch?v=MRXNcrGkr4Y
    I like the concept and its very nicely done!
    Good stuff Billy. You owned this.+1

    Comment by admin — 28 March 2012 @ 12:20 pm

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
(c) 2014 Interactive Art and Computational Design, Spring 2012 | powered by WordPress with Barecity