How to: Convert an OpenCV cv::Mat to an OpenGL texture

I’m working on using OpenCV to get Kinect sensor data via OpenNI, and needed a way to get a matrix (cv::Mat) into an OpenGL texture – so I wrote a function to do just that – woo! Apologies in advance for the terrible juggling ;-)

The function used to perform the sensor data to texture conversion is:

// Function turn a cv::Mat into a texture, and return the texture ID as a GLuint for use
GLuint matToTexture(cv::Mat &mat, GLenum minFilter, GLenum magFilter, GLenum wrapFilter)
{
	// Generate a number for our textureID's unique handle
	GLuint textureID;
	glGenTextures(1, &textureID);
 
	// Bind to our texture handle
	glBindTexture(GL_TEXTURE_2D, textureID);
 
	// Catch silly-mistake texture interpolation method for magnification
	if (magFilter == GL_LINEAR_MIPMAP_LINEAR  ||
	    magFilter == GL_LINEAR_MIPMAP_NEAREST ||
	    magFilter == GL_NEAREST_MIPMAP_LINEAR ||
	    magFilter == GL_NEAREST_MIPMAP_NEAREST)
	{
		cout << "You can't use MIPMAPs for magnification - setting filter to GL_LINEAR" << endl;
		magFilter = GL_LINEAR;
	}
 
	// Set texture interpolation methods for minification and magnification
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, minFilter);
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, magFilter);
 
	// Set texture clamping method
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, wrapFilter);
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, wrapFilter);
 
	// Set incoming texture format to:
	// GL_BGR       for CV_CAP_OPENNI_BGR_IMAGE,
	// GL_LUMINANCE for CV_CAP_OPENNI_DISPARITY_MAP,
	// Work out other mappings as required ( there's a list in comments in main() )
	GLenum inputColourFormat = GL_BGR;
	if (mat.channels() == 1)
	{
		inputColourFormat = GL_LUMINANCE;
	}
 
	// Create the texture
	glTexImage2D(GL_TEXTURE_2D,     // Type of texture
	             0,                 // Pyramid level (for mip-mapping) - 0 is the top level
	             GL_RGB,            // Internal colour format to convert to
	             mat.cols,          // Image width  i.e. 640 for Kinect in standard mode
	             mat.rows,          // Image height i.e. 480 for Kinect in standard mode
	             0,                 // Border width in pixels (can either be 1 or 0)
	             inputColourFormat, // Input image format (i.e. GL_RGB, GL_RGBA, GL_BGR etc.)
	             GL_UNSIGNED_BYTE,  // Image data type
	             mat.ptr());        // The actual image data itself
 
	// If we're using mipmaps then generate them. Note: This requires OpenGL 3.0 or higher
	if (minFilter == GL_LINEAR_MIPMAP_LINEAR  ||
	    minFilter == GL_LINEAR_MIPMAP_NEAREST ||
	    minFilter == GL_NEAREST_MIPMAP_LINEAR ||
	    minFilter == GL_NEAREST_MIPMAP_NEAREST)
	{
		glGenerateMipmap(GL_TEXTURE_2D);
	}
 
	return textureID;
}

You can then use the above function like this:

// Create our capture object
cv::VideoCapture capture( CV_CAP_OPENNI );
 
// Check that we have actually opened a connection to the sensor
if( !capture.isOpened() )
{
	cout << "Cannot open capture object." << endl;
	exit(-1);
}
 
// Create our cv::Mat object
cv::Mat camFrame;
 
	// *** loop ***
 
	// Grab the device
	capture.grab();
 
	// Retrieve desired sensor data (in this case the standard camera image)
	capture.retrieve(camFrame, CV_CAP_OPENNI_BGR_IMAGE);
 
	// Convert to texture
	GLuint tex = matToTexture(camFrame, GL_NEAREST, GL_NEAREST, GL_CLAMP);
 
	// Bind texture
	glBindTexture(GL_TEXTURE_2D, tex);
 
	// Do whatever you want with the texture here...
 
	// Free the texture memory
	glDeleteTextures(1, &tex);
 
	// *** End of loop ***
 
// Release the device
capture.release();

There’s one very important issue to watch out for when using OpenCV and OpenNI together which I’ve commented in the code, but I’ll place here as well as it can be a real deal breaker:

There appears to be a threading issue with the OpenCV grab() function where if you try to grab the device before it’s ready to provide the next frame it takes up to 2 seconds to provide the frame, which it might do for a little while before crashing the XnSensorServer process & then you can’t get any more frames without restarting the application. This results in horrible, stuttery framerates and garbled sensor data.

I’ve found that this can be worked around by playing an mp3 in the background. No, really. I’m guessing the threading of the mp3 player introduces some kind of latency which prevents the grab() function being called too soon. Try it if you don’t believe me!

So just be aware that if you’re using a Kinect you have to be careful with the grab() function… The source code used to create the above video is provided in full after the jump, if you’re interested.

Cheers!

Continue reading How to: Convert an OpenCV cv::Mat to an OpenGL texture

How To: Convert VirtualPC .vhd hard drives to VirtualBox .vdi hard drives (or vice-versa)

Don’t bother – VirtualBox now natively supports .vhd drives – how handy is that? =D

But if you really wanted to, just use:

VBoxManage clonehd source.vhd target.vdi --format vdi

This will place the .vdi version of your drive in your current users VirtualBox folder, which is: ~/.VirtualBox

Relatedly, to go the other way (.vdi to .vhd) you could use:

VBoxManage clonehd source.vdi target.vhd --format vhd

Note: For this to work, you must not have the drive attached to any virtual machine, so if you already have it connected to a virtual machine, detach it first, mkay?

How To: Convert .WBFS Image Files to .ISO Images

Update: You can convert compressed ISO images (.ciso files) using this method, too…

If you need to translate a Wii DVD image from the newer .WBFS format (which has lovely sparce-file support) to an old-school .ISO image, well you can jolly well do so with Wii Backup Manager! Now, WBM is a really useful program so congrats to the author (xzxero) – but it uses non-standard GUI elements in that it uses what looks like the category list as a second main-menu, and this makes it a little non-intuitive to use. But in fairness to the author, the download page has a tutorial (which I didn’t find until after completing this post!). Still, it’s easy enough once you’ve figured that out. The entire conversion process goes like this:

1.) Grab yourself a copy of Wii Backup Manager. Version v0.3.5 beta 1 is the most recent version available at the time of writing, and although it’s Win32 only, it works fine in VirtualBox…

2.) Extract and launch it

Wii Backup Manager

3.) From the Files tab, click on what looks like the Add category title, select Files and go pick a .WBFS image from the navigation dialogue box

Wii Backup Manager - Add Files

4.) Still in the Files tab, tick the checkbox to the left of your .wbfs image, and then click on what looks like the Transfer category title and select ISO..

Wii Backup Manager - Transfer Files

5.) Pick where you want the converted .ISO file to be created and click the [OK] button

Wii Backup Manager - Place ISO

That’s it – job done! Transfer your ISO to your WBFS formatted USB drive and launch it as you normally would via USBLoaderGX or whatever your poison is… :)
Flattr this

How To: Easily Convert FLAC audio to MP3s in Linux

I grabbed a bunch of FLAC files the other day, and as nice as they sound, I don’t think ~30-40MB per track is acceptable, so I did a bit of research and stumbled across this great post on LinuxTutorialBlog.

Turns out there’s a dead simple GUI based tool called SoundConverter – which really is as simple as pointing it at a directory and configuring your transcoding preferences (mp3, ogg, file-naming etc). A swift sudo apt-get install soundcoverter and a couple of clicks later and the job’s done.

SoundConverter1SoundConverter2

If you really want a bash method, there’s a bunch of scripts and links in tfa, such as this one by Octavio Ruiz :

#!/bin/bash
#
# Copyright 2008 Octavio Ruiz
# Distributed under the terms of the GNU General Public License v3
# $Header: $
#
# Yet Another FLAC to MP3 script
#
# Author:
# Octavio Ruiz (Ta^3) <tacvbo@tacvbo.net>
# Modification/Fixage:
# r3dux
# Thanks:
# Those comments at:
# http://www.linuxtutorialblog.com/post/solution-converting-flac-to-mp3
# WebPage:
# https://github.com/tacvbo/yaflac2mp3/tree
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY. YOU USE AT YOUR OWN RISK. THE AUTHOR
# WILL NOT BE LIABLE FOR DATA LOSS, DAMAGES, LOSS OF PROFITS OR ANY
# OTHER KIND OF LOSS WHILE USING OR MISUSING THIS SOFTWARE.
# See the GNU General Public License for more details.
 
LAME_OPTS="-V 0 -q 0 --vbr-new"
 
id3v2=$(which id3v2)
old_IFS=${IFS}
IFS=''
files=( `find . -type f -name '*flac'` )
 
for N_files in ${!files[@]}
do
vars=( `metaflac --no-utf8-convert --export-tags-to=- "${files[${N_files}]}"` )
 
for N_vars in ${!vars[@]}
do
    export "$(echo "${vars[${N_vars}]%=*}" | tr [:upper:] [:lower:])=${vars[${N_vars}]#*=}"
done
 
flac -dc "${files[${N_files}]}" |\
    lame ${LAME_OPTS} --ignore-tag-errors --add-id3v2 \
        ${artist:+--ta} ${artist} \
        ${tracknumber:+--tn} ${tracknumber} \
        ${title:+--tt} ${title} \
        ${album:+--tl} ${album} \
        ${date:+--ty} ${date} \
        ${genre:+--tg} ${genre} \
        ${comment:+--tc} ${comment} \
        - "${files[${N_files}]/\.flac/.mp3}"
 
    [[ -x ${id3v2} ]] && ${id3v2} \
        ${artist:+--artist} ${artist} \
        ${tracknumber:+--track} ${tracknumber} \
        ${title:+--song} ${title} \
        ${album:+--album} ${album} \
        ${date:+--year} ${date} \
        ${genre:+--genre} ${genre} \
        ${comment:+--comment} ${comment} \
        "${files[${N_files}]/\.flac/.mp3}"
 
done
IFS=${old_IFS}

I’ve modded the LAME_OPTS line in the above script to use the -q 0 switch in lame (so it uses the highest quality algorithm it can), and changed the order of when the ${LAME_OPTS) options are passed to lame, which results in them actually being honoured. Which is nice. Should you have any specific encoding goals, you can always browse through the lame switches and mod it to your hearts content.

Sweet like chocolate =D

Note: To run the above script, just copy & paste into a file, maybe flac2mp3.sh or something, then chmod +x flac2mp3.sh to make it executable and run it on a folder like flac2mp3.sh MyFolderOfFLACFiles.