Multiple Hand Tracking

This program uses the depth sensors to track multiple hands. SimpleOpenNI version 1.96 allows for easy multiple hand tracking compared to older versions. This program will display the vector coordinates of each hand tracked and display a colored circle at those positions. One thing to watch for is that the tracked hand’s ID number starts at one so you have to compensate for starting at one when using arrays (since they start at index 0).

import SimpleOpenNI.*;
// kinect object
SimpleOpenNI kinectObject;

// kinect image
PImage kinectImage;

// vector position of hands tracked
ArrayList handPositions;
// array of colors to used for eadch tracked hand
color[] handColor = new color[]{

//size of dot that will show tracked hand's position
int dotSize = 10;
Creates kinect object and draw window.  Enables hand gesture and tracking.
Adds empty vector to vector array since hand id# starts at one.
void setup(){
	  //create kinect object
	  kinectObject = new SimpleOpenNI(this);
	  // enable depthMap and RGB camera
	  //create window the size of kinect resolution
	  // disable mirror image
	  // enable hands and gesture
	  // create a new array for position of hands
	  handPositions = new ArrayList();
	  // hand id # starts at one - fill zero index with blank vector
	  PVector tempPos = new PVector(0,0,0);
Updates kinect image and draws to screen.  For each hand position tracked,
print out coordinates and draw ellipse on hand.
void draw(){
	  // update the cam
	  // get the kinect image from the RGB camera
	  kinectImage = kinectObject.rgbImage();
	  // display the image on the screen
	  // for each hand being tracked, display each hands vector postion
	  for(int i=1;i<handPositions.size();i++){

	    //get the current ahnd's position
	    PVector tempVector = handPositions.get(i);

	    //print out current hand's id# and position
	    println("handId: " + i + ", position: " + tempVector);

	    // change draw color based on hand id# (since hand id# starts at one
	    // get the previous color array index

	    // fill the ellipse with the same color

	    // draw ellipse at x/y coordinates given ellipse size
When a new hand is found, print out hand id and add position to vector array
void onNewHand(SimpleOpenNI curContext,int handId,PVector pos){
  //print out new tracked hand id #
  println("new hand Id: " + handId);
  // add current position of hands to vector arraylist
While hand is being tracked, update new vector coordinates.
void onTrackedHand(SimpleOpenNI curContext,int handId,PVector pos){
  // convert vector so that top left corner is (0,0)
  curContext.convertRealWorldToProjective(pos, pos);
  // update vector positions of hand in vector arraylist
When tracked hand is lost then display hand of lost hand and remove
from vector array.
void onLostHand(SimpleOpenNI curContext,int handId){
  // print out that hand was lost
  println("hand lost - hand Id: " + handId);
  // remove hand vector from vector arraylist
When hand gesture is found, Start tracking hand then print out vector coodinates
of where hand was found and its hand id #.
void onCompletedGesture(SimpleOpenNI curContext,int gestureType, PVector pos){
  // initiate hand tracking and give hand id number starting to track
  int handId = kinectObject.startTrackingHand(pos);
  // print out hand is starting to track, what gesture used, and where it was found
  println(" hand tracking - gestureType: " + gestureType + ", vector position: " + pos + " hand id: " + handId);

Leave a Reply

Your email address will not be published. Required fields are marked *