How to: Convert an OpenCV cv::Mat to an OpenGL texture

I’m working on using OpenCV to get Kinect sensor data via OpenNI, and needed a way to get a matrix (cv::Mat) into an OpenGL texture – so I wrote a function to do just that – woo! Apologies in advance for the terrible juggling ;-)

The function used to perform the sensor data to texture conversion is:

You can then use the above function like this:

There’s one very important issue to watch out for when using OpenCV and OpenNI together which I’ve commented in the code, but I’ll place here as well as it can be a real deal breaker:

There appears to be a threading issue with the OpenCV grab() function where if you try to grab the device before it’s ready to provide the next frame it takes up to 2 seconds to provide the frame, which it might do for a little while before crashing the XnSensorServer process & then you can’t get any more frames without restarting the application. This results in horrible, stuttery framerates and garbled sensor data.

I’ve found that this can be worked around by playing an mp3 in the background. No, really. I’m guessing the threading of the mp3 player introduces some kind of latency which prevents the grab() function being called too soon. Try it if you don’t believe me!

So just be aware that if you’re using a Kinect you have to be careful with the grab() function… The source code used to create the above video is provided in full after the jump, if you’re interested.

Cheers!

Continue reading How to: Convert an OpenCV cv::Mat to an OpenGL texture

New XBox 360 Motion Capture System

Looks interesting, and I really don’t want to be a hater, but you’ve got to wonder about some things:

– How fine a degree of control will you have?
– If I’m playing a skateboarding game, can the camera tell if I want the board to do a heelflip, or a heelflip shove-it? Ya know, is it accurate?
– How accurate do -I- have to be?
– How knackering will playing fighting games be on a scale of 1 to fuck-that?
– How’s a FPS gonna work? One hand for gun angle and the other for forward/back/left/right/strafe?
– Who do I sue when I break my neck on the coffee table playing Mirrors Edge 2?

Also, it’s all showing an idealised version of things not real gameplay, as the kids fingers were covering part of the board when he scanned it, yet no fingers in the scanned image. Hmm, I’m coming off as a hater here – and I’m sure lots of cool uses for this will arise, I’m just not sure about how many of them aren’t kinda gimicky…

Also, the voice recognition will be like this – no matter WHAT you ask it to do =P :

http://www.youtube.com/watch?v=0H5k–n7sFI

Okay, so I’m cynical as all hell AND a playah-hater… Oh, well. I really hope it works out, but I guess the only way to find out will be to wait and see…