How to: Convert an OpenCV cv::Mat to an OpenGL texture

I’m working on using OpenCV to get Kinect sensor data via OpenNI, and needed a way to get a matrix (cv::Mat) into an OpenGL texture – so I wrote a function to do just that – woo! Apologies in advance for the terrible juggling ;-)

The function used to perform the sensor data to texture conversion is:

You can then use the above function like this:

There’s one very important issue to watch out for when using OpenCV and OpenNI together which I’ve commented in the code, but I’ll place here as well as it can be a real deal breaker:

There appears to be a threading issue with the OpenCV grab() function where if you try to grab the device before it’s ready to provide the next frame it takes up to 2 seconds to provide the frame, which it might do for a little while before crashing the XnSensorServer process & then you can’t get any more frames without restarting the application. This results in horrible, stuttery framerates and garbled sensor data.

I’ve found that this can be worked around by playing an mp3 in the background. No, really. I’m guessing the threading of the mp3 player introduces some kind of latency which prevents the grab() function being called too soon. Try it if you don’t believe me!

So just be aware that if you’re using a Kinect you have to be careful with the grab() function… The source code used to create the above video is provided in full after the jump, if you’re interested.


Continue reading How to: Convert an OpenCV cv::Mat to an OpenGL texture

How to: manipulate webcam streams with OpenCV and OpenGL

Okay – third and final part… I saw a video, I wanted to recreate the effect they used, and now I have, on a live, real-time-processed webcam stream using C++, OpenCV and OpenGL. Yay! =D

Full source code after the jump =D

Continue reading How to: manipulate webcam streams with OpenCV and OpenGL

How to: Pixelise a webcam stream using OpenCV

There was a video the other day which I posted about where the video footage was all highly pixelised into circles of varying sizes and colours, and I reckoned I could produce a similar effect by either resizing the stream down so it’s really blocky then scaling it back up or reading all the pixels in a block, averaging the colour and then drawing blocks of that averaged colour.

Well, I had an hour or so today to do a bit of “me-coding”, and in the end I took the second option.

Pixelised Webcam Stream

The pixelisation works on the live stream, and you can drag the slider around to switch through from 1 division (i.e. the entire window is one block of solid colour) to 160 divisions.

I’ll bring the values into OpenGL and see what I can do with points and the like when I have another hour or two spare over the coming few days – fun stuff =D

Source code after the jump for those interested…

Continue reading How to: Pixelise a webcam stream using OpenCV

How To: Perform Real-Time OpenCV Edge Detection on a WebCam Stream

I’ve started learning OpenCV, the open-source computer vision library, and I’ve got to say – it’s absolutely brilliant. In the below screenshot I’m just running the canny edge detection filter on the live stream and outputting it in a window with some sliders which link to the edge detection parameters, and the entire thing from initialising the web cam, to displaying the frames and performing all the filtering is a little over 100 lines of code… Amazing!

OpenCV Canny Edge Detection

It’s when I start asking OpenCV to do things that it currently doesn’t do that I’m going to start screaming, and I read that optical flow analysis has stolen the youth of many a Ph.D student… But how awesome will it be if I can make a really robust gesture recognition system and tie it into OpenGL? I guess there’s only one way to find out…

Full source code after the jump for those interested (although please don’t ask me how to build/install OpenCV and link the libraries into the project – it’ll vary depending on your system and build tools of choice).

Continue reading How To: Perform Real-Time OpenCV Edge Detection on a WebCam Stream