DMGordon – Place

For my place project, I made a point cloud compositing system which condenses multiple point clouds into a single three dimensional pixel grid.

I began this project with the intent of capturing the freeway I live next to and a creek in Frick Park. I started with attempting to generate point clouds using a stereoscopic video camera and Kyle McDonald’s OFCV OpenFrameworks addon. The resulting images were too noisy for compositing, however, and I had to switch capture devices to the Kinect. While the Kinect provides much cleaner point clouds, it requires an AC outlet for additional power, tethering me to wall sockets. Others have found portable power supplies to remedy this, and my next step is to follow their lead in making a mobile capture rig.

The Kinect automatically filters its infrared and visible color data through its own computer vision algorithms to produce denoised depth data, which can then be projected onto the visible color image to create colored, depth-mapped images. I could then take each pixel of these images and use the depth data as a z-coordinate to populate a three dimensional color grid. By adding multiple frames of a depth-mapped video into a single color grid, we are treating the color grid like a piece of photopaper during a long exposure. The resulting images contain the familiar effects of long exposure photography in a three dimensional vessel:

In addition to containing an additional dimension of space, we can run colorpoint-by-colorpoint analysis on the grid to split and extract subsections of the image based on various factors. Here are some tests experimenting with simple color-range partitioning:

Full Spectrum:

Filtered for Greenness:

Filtered for Not-Greenness:

Going forward, I see several ways to expand these methods into provocative territory. With a portable capture rig, we could capture more organic and active locations, where movement and changes in light could lead to more dynamic composites. Also, more intense analytics on the composites, such as filtering blocks of the images based on facial recognition or feature classification, would produce more compelling cross sections of the data. Adding multiple Kinects compositing their point clouds into multi-perspective frames would open the door to volumetric analysis. Even rigging a single Kinect up to a Vive controller or an IMU could provide a global coordinate space for expanding the composites beyond a single frustrum of reference.

Here are a couple more images of backyard experiments:

BBQ + House true-color:

BBQ + House 360:

More BBQ related pictures to come