CV: Color tracking

In the blob tracking example, you’ve learned a basic brightness tracking algorithm. In this recipe, we’ll up the ante and write an algorithm to track colored pixels. This technique will be useful if you want to create an installation that more than one person can interact with. For instance, you can give each participant a brightly colored ball they can use to wave at the camera to control something on the screen. The code for this example is similar to the blob tracking sketch from the previous recipe. The only difference is that we need a color variable named trackColor that we’ll use to track, and three integer variables, each for its red, green, and blue values. The threshold variable is gone, and we use an integer variable named maxColorDifference, instead. import processing.video.*; Capture webcam; color trackColor; int trackR; int trackG; int trackB; int topLeftX; int topLeftY; int bottomRightX; int bottomRightY; int maxColorDifference; void setup() { size( 640, 480 ); webcam = new Capture( this, width, height ); webcam.start(); trackColor = color( 255 ); trackR = (trackColor >> 16) & 0xff; trackG = (trackColor >> 8) & 0xff; trackB = trackColor & 0xff; maxColorDifference = 40; topLeftX = width; topLeftY = height; bottomRightX = 0; bottomRightY = 0; } Inside the draw() function, we’ll use a similar technique to the brightness tracking algorithm. Instead of using the brightness() function, we’ll separate the color of each pixel into a red, green, and blue value and calculate the distance between those with the dist() function. This distance is then compared to the maxColorDifference variable, so that we can calculate the bounding box around the colored pixels. void draw() { if ( webcam.available() ) { webcam.read(); image( webcam, 0, 0 ); loadPixels(); int counter = 0; for ( int j = 0; j < webcam.height; j++ ) { for ( int i = 0; i < webcam.width; i++ ) { color c = webcam.pixels[counter]; int r = (c >> 16) & 0xff; int g = (c >> 8) & 0xff; int b = c & 0xff; float colorDifference = dist( r, g, b, trackR, trackG, trackB ); if ( colorDifference < maxColorDifference ) { if ( i < topLeftX ) { topLeftX = i; } if ( j < topLeftY ) { topLeftY = j; } if ( i > bottomRightX ) { bottomRightX = i; } if ( j > bottomRightY ) { bottomRightY = j; } } counter++; } } updatePixels(); // draw tracking color fill( trackColor ); noStroke(); rect( 0, 0, 20, 20 ); noFill(); stroke( 0 ); strokeWeight( 2 ); rect( topLeftX, topLeftY, bottomRightX – topLeftX, bottomRightY – topLeftY ); // reset tracking points topLeftX = width; topLeftY = height; bottomRightX = 0; bottomRightY = 0; } } The mousePressed() function is used to set the track color. Hold a brightly colored object in front of the camera, and click on it with the mouse. void mousePressed() { trackColor = webcam.get( mouseX, mouseY ); trackR = (trackColor >> 16) & 0xff; trackG = (trackColor >> 8) & 0xff; trackB = trackColor & 0xff; } If all goes well, you’ll see how a bright colored object is recognized. The only important difference in the brightness tracking algorithm from the previous recipe is the following piece of code: color c = webcam.pixels[counter]; int r = (c >> 16) & 0xff; int g = (c >> 8) & 0xff; int b = c & 0xff; float colorDifference = dist( r, g, b, trackR, trackG, trackB ); We’ll take the color of the current pixel, and store it in a variable named c. We separate this color into its red, green, and blue components, with a technique called bit shifting. The line int r = (c >> 16) & 0xff; does the same as int r = red( c ) does but is a lot faster. The following lines do the same as the green() and blue() functions. This code is a little hard to read, but it will give your sketch a small speed bump, when you iterate over a lot of pixels. You can learn more about this technique in the Processing reference at http://processing.org/reference/rightshift.html. The dist() function is usually used to calculate the distance between two points in a 2D or 3D space. I used the red, green, and blue components from the colors as the x, y and z coordinates of a point in a 3D space. The dist() function is used in this case to calculate the difference between the tracking color and the color of the current pixel. The smaller this number, the more similar the colors are. If the value of the colorDifference variable is smaller than the value of the maxColorDifference variable, the color is similar enough to the tracking color and can be used to calculate the bounding box. The algorithm to calculate the top, left-hand side and bottom, right-hand side coordinates of this box is the same as in the brightness tracking example.

The best way to predict the future is to invent it (Alan Kay)