Lecture 11-02-09

by Dan Wilcox @ 9:14 am 9 February 2011

Today we are setting up openFrameworks to work with the MS Kinect depth camera.

Install

Grab OF from openframeworks.com/downloads and the following OF Kinect addons:

Note: These addons currently only work for Mac and Linux (ofxKinect). We will be working on getting Windows versions shortly.

Uncompressing the openFrameworks zip gives you the OF folder layout:

/addons   --- where to place ofx addons
/apps       --- apps be here
/libs
/other
/scripts

Uncompress ofxKinect and ofxOpenNI and place them in the /addons folder.

Building and Running Apps

You can try any of the example apps in /apps/examples or /example/addonsExamples.

Open the project file in the  an example folder with you environment of choice:

  • Mac: XCode .xcodeproj
  • Linux: Codeblocks .workspace
  • Windows: Visual Studio .sln, Codeblocks .workspace

Build the example and run it. Some examples have 2 project files, one for OF 0062 and one for OF 007 beta. Make sure to use the project file for the correct OF version you have.

Note: If you are using XCode and don’t like how it uses multiple windows, I recommend using the All-In-One layout. Set this in Preferences by:

XCode->Preferences->General->Layout->All-In-One

If the Layout dropdown box is grey, close all the XCode windows then open the preferences again.

Creating a New App

The easiest way to create a new app project is to copy an example to your apps folder. The /apps/examples/emptyExample project is a bare bones project which is a good start. Copy the emptyExample folder into /apps/myApps.

To rename to project on Mac:

  • rename the project folder
  • open the project in XCode and rename it in Projects->Rename

If you want to use an addon, look for one of the addon examples. ofxKinect and ofxOpenNI for instance have example project folders which you can use to start.

Adding an Addon to an Existing App

On Mac:

  • in XCode, create a new Group for the addon ie “ofxOpenCV” by right clicking on addons under the source tree and selecting Add->New Group
  • drag the src and lib  directories from the addon folder in the Finder into the Group you created in XCode

Some external (ofx*) libraries have special instructions such as ofxOpenNI in their README files. Make sure to double check these when using addons which are not included with OF by default.

2 Comments »

  1. Here is some simple code that I was using last semester to play with recorded Kinect images.

    NOTE: Updated for current ofxKinect. Dan Wilcox

    You can record to an iamge sequence and load it later for offline usage. This is very simple, be careful not to record/playback too many! Check your data folder to make sure you don’t fill it too full.

    You will need ofxDirList. Code is as follows… (note: not fully tested, treat as psuedo code)

    testApp.h:

    
    #include "ofxDirList,h"
    
    // comment this to use the recorded images
    #define KINECT_CONNECTED
    
    // uncomment this to record
    //#define KINECT_RECORD
    
    ...
    
    	void exit();
    
    ...
    
    	ofxKinect kinect;
    	ofImage depthImage;	// the greyscale depth image form the kinect (or recording)
    
    	ofxDirList dir; 		// directory lister
    	int saveCounter;		// saved image index for recording
    	vector savedImages;	// the saved images for playback
    	int currentSavedImage;	// current image index
    

    testApp.cpp:

    void testApp::setup() {
    	
    	ofSetFrameRate(30);
    
    	depthImage.allocate(kinect.width, kinect.height, OF_IMAGE_GRAYSCALE);
    
        // using a live kinect?
        #ifdef KINECT_CONNECTED
    
    	kinect.init();
    	kinect.open();
    	
        // not connected
    	#else 
    		
    	// Load png files from file
    	dir.allowExt("png");
    	dir.setVerbose(false);
    	string folder = "01/";
    	totalImages = dir.listDir(folder);
    	saveCounter = 0;
    	currentImage = 0;
    
    	// load all recorded depth images in "data/01/"
    	for(int i = 0; i < totalImages; i++){
    		ofImage *img = new ofImage();
    		img.loadImage(folder + dir.getName(i));	
    		img.setImageType(OF_IMAGE_GRAYSCALE); 
    		savedImages.push_back(img);
    	}
    
    	#endif
    }
    
    void testApp::update()
    {
    
    	// load a saved image for playback
    	#ifdef KINECT_CONNECTED
    		ofImage *img = savedImages[currentImage];
    		depthImage.setFromPixels(img->getPixels(),
    			img->getWidth(), img->getHeight, OF_IMAGE_GRAYSCALE);
    	#endif
    
    	// nothing will happen here if the kinect is unplugged
    	kinect.update();
    	if(kinect.isFrameNew())	// there is a new frame and we are connected
    	{
    		// grab the depth image from the kinect
    		depthImage.setFromPixels(kinect.getDepthPixels(),
    			kinect.width, kinect.height, GL_LUMINANCE);
    		
    		// record the depth images to "data/01/" as long as we are running
    		#ifdef KINECT_RECORD
    			depthImage.saveImage("01/"+ofToString(saveCounter)+".png");
    			saveCounter++;
    		#endif
    
    ...
    
    	}
    
    	// do some cv using depthImage here
    	
    ...
    
    }
    
    void testApp::exit()
    {
    	// cleanup any loaded images
    	for(int i = 0; i < savedImages.size(); i++){
    		ofImage *img = savedImages[i];
    		delete img;
    	}
    	savedImages[i].clear();
    }
    
    Comment by Shawn Sims — 9 February 2011 @ 11:16 am
  2. Here’s how to access the skeleton positions with ofxOpenNI:

    // get the first user skeleton
    ofxTrackedUser* tracked = user.getTrackedUser(0);
    
    if(tracked != NULL)
    {
        // get the upper left shoulder pos
        ofxPoint pos = tracked->left_shoulder.end
    }

    Check the header files for more info. The interface is very rough for now, so you will have to figure some things out for yourself.

    Comment by Dan Wilcox — 9 February 2011 @ 11:47 am

Leave a comment

You must be logged in to post a comment.

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
(c) 2017 Interactive Art & Computational Design / Spring 2011 | powered by WordPress with Barecity