The Leap motion input device looks incredible

The Leap is a new 3D input motion controller in a similar vein to the Kinect, but with apparently about 200x the resolution – how amazing is that?

To quote the Leap people:

Leap represents an entirely new way to interact with your computers. It’s more accurate than a mouse, as reliable as a keyboard and more sensitive than a touchscreen. For the first time, you can control a computer in three dimensions with your natural hand and finger movements.

I’ve applied for a free dev-kit, so fingers-crossed! Worst case though, the sensor’s only meant to be around the $70 mark when they come out.

Would so love to work with this tech for my Ph.D research… looks like it solves a lot of problems right off the bat. Awesome!

Voice recognition – we’re not quite there yet

Voice recognition is only ever going to get better (check out CMU Sphinx if you want to play with speech rec. for yourself), but at the moment… um, it’s not quite up to scratch:

Okay, so he’s putting in on a bit for the lulz, and if he’d just said “where’s the nearest pub?” he’d probably get a useful answer – but that’s half the battle with speech recognition. At the moment we have to adapt our speech to the software, but in the future I don’t think we’ll have to at all.

At least for the time being, as Penny Arcade put it, it’s all going to be a bit like this:

Penny Arcade Kinect Integration - Jan-2012

Best keep your deer-combs handy.

How to: Convert an OpenCV cv::Mat to an OpenGL texture

I’m working on using OpenCV to get Kinect sensor data via OpenNI, and needed a way to get a matrix (cv::Mat) into an OpenGL texture – so I wrote a function to do just that – woo! Apologies in advance for the terrible juggling ;-)

The function used to perform the sensor data to texture conversion is:

You can then use the above function like this:

There’s one very important issue to watch out for when using OpenCV and OpenNI together which I’ve commented in the code, but I’ll place here as well as it can be a real deal breaker:

There appears to be a threading issue with the OpenCV grab() function where if you try to grab the device before it’s ready to provide the next frame it takes up to 2 seconds to provide the frame, which it might do for a little while before crashing the XnSensorServer process & then you can’t get any more frames without restarting the application. This results in horrible, stuttery framerates and garbled sensor data.

I’ve found that this can be worked around by playing an mp3 in the background. No, really. I’m guessing the threading of the mp3 player introduces some kind of latency which prevents the grab() function being called too soon. Try it if you don’t believe me!

So just be aware that if you’re using a Kinect you have to be careful with the grab() function… The source code used to create the above video is provided in full after the jump, if you’re interested.

Cheers!

Continue reading How to: Convert an OpenCV cv::Mat to an OpenGL texture

Child of Eden

New on-rails-rhythm-shooter-thingy Child of Eden (from Tetsuya Mizuguchi the same guy who designed Rez, no less) is looking pretty darn nifty, especially with Kinect controls. Though quite how long you’ll last without your arms getting tired is quite another matter.

If you’ve never played or heard of Rez – it has to be one of the most beautiful and original games ever made. Originally for the Dreamcast, it has a HD remake on the 360 (via XBox Live Arcade), and is just great. Also, it has one of the easiest control schemes ever, meaning pretty much anyone can play it even if they’re not used to game controllers. You move the targeting reticule around (up/down/left/right), and hold down or hammer the fire button. If you’re holding the button down, anything that passes through the reticule is targeted (up to 8 things, or 8 shots on the same thing, or any combination), and when you release the button, up to eight shots shoot off and hit stuff. All perfectly timed to each levels unique techno/house/ambient soundtrack. Brilliant!

See? If Child of Eden is Rez 2.0, then I definitely want some :)

Also, and I only just noticed this – but I posted about the track California Soul by Marlena Shaw (or a remix thereof) the other day – and guess what the music in the latter half of the Rez video samples? I knew I’d heard it somewhere before! How’s that for serendipity?