Processing Month, Day 31 - Building Interactive Installations with OpenCV

Posted on 2011-05-31 by Jan Vantomme
Tags: processing, tutorial

For the last day in my Processing Month series, I'm going to cover the basics of building interactive installations with Processing. The name of the installation we'll build is "The Blurry Mirror". The concept is easy: the computer screen will act as a digital mirror. The closer somebody gets to the installation, the more the image in the digital mirror will be blurred. We're going to do some facetracking with the OpenCV library for Processing. You should download and install the OpenCV library to get started.

Setting up OpenCV

Inside the setup() function, we need to setup OpenCV and tell it what we're going to track. I have set OpenCV to capture the video from my webcam at a resolution of 640 x 480 pixels. This should be enough for most installations. You can use a higher resolution if your camera supports it, but this will require more processing time for the CPU. The last OpenCV method I used in the setup() of my sketch is cascade(). I'm using one of the standard cascades (CASCADE_FRONTALFACE_ALT) to track a face in a video. This method can also be used to load a custom xml file with a description for tracking objects.

size( 1024, 768 );
opencv = new OpenCV( this );
opencv.capture( 640, 480 );
opencv.cascade( OpenCV.CASCADE_FRONTALFACE_ALT );

Preparing the Image

The next thing we need to do is to prepare the image from the camera for face tracking. This code below should be the first part inside the draw() function. The read() method will get the current image from the camera so we can use it in our installation. We also need to flip our image with FLIP_HORIZONTAL to get the mirror effect. You can also adjust the contrast and brightness of the image to get better tracking when needed. These methods need an integer between -128 and 128. I've used the arrow keys to adjust the values. You can display the image by using the OpenCV image() method as the first parameter for the standard Processing image() function.

opencv.read();
opencv.flip( OpenCV.FLIP_HORIZONTAL );
opencv.contrast( contrastValue );
opencv.brightness( brightnessValue );
image( opencv.image(), 0, 0 );

Detecting Faces

Now that our image is prepared, we need to use the detect() method to scan the image for frontal faces. We can do this with the following line of code. I've imported java.awt.Rectangle because the detect() method will return an array of rectangles. These rectangles have a location, width and height we can use as parameters for blurring the final output of our installation.

Rectangle[] faces = opencv.detect( 1.2, 2, OpenCV.HAAR_DO_CANNY_PRUNING, 40, 40 );

Because a mirror is something personal, I only want to track one face, so only one person at a time will be able to interact with the installation. The first if statement is really important. We only need to do things with the image or the rectangles if a face is found in the image, otherwise the installation would crash. I've mapped the coordinate and the width of the first face to the size of our Processing sketch. The variable f is then used to decide how much the final image should be blurred. If the size of the face is small (far away from the camera), the image doesn't get blurred, if the size is big (close to the camera) the image gets blurred a lot. I also added an if statement for debugging. This can be toggled by pressing the d key on your keyboard. The image below shows you the blurred image when somebody gets close to the camera.

if ( faces.length > 0 ) {
    // values needed for debugging
    float x = map( faces[0].x, 0, 640, 0, 1024 );
    float y = map( faces[0].y, 0, 480, 0, 768 );
    float f = map( faces[0].width, 0, 640, 0, 1024 );       
    if ( f <= 200) {
        // no blurring
    } else if ( f > 200 && f <= 300 ) {
        opencv.blur( OpenCV.GAUSSIAN, 11 );    
    } else if ( f > 300 && f <= 400 ) {
        opencv.blur( OpenCV.GAUSSIAN, 31 );    
    } else if ( f > 400 && f <= 500 ) {
        opencv.blur( OpenCV.GAUSSIAN, 51 );   
    } else {
        opencv.blur( OpenCV.GAUSSIAN, 71 );
    }
    image( opencv.image(), 0, 0, width, height );
    if ( DEBUG ) {
        noFill();
        stroke( 255, 0, 0 );
        rect( x, y, f, f );
    }
}

The last thing we need to do is to add an extra function to our application to stop OpenCV when our application quits.

public void stop()
{
    opencv.stop();
    super.stop();
}

Download

We covered the basics of OpenCV, so you should be able to get started and build your own interactive installation. You can download the example for the Blurry Mirror installation and experiment with it.

Tweet this article