How-To: Replace characters in filenames in Linux

Occassionally you’re likely to end up with a bunch of files with filenames that are underscore separated or space separated when you want them the other way around, and then you either have to manually rename them by hand, or live with it.

I had this issue the other day so I wrote a script to convert all filenames in the current directory to the specified format.

The script switches are:

  • -t – turns on [T]esting mode, which displays what would happen if you ran the script, but makes absolutely no changes to filenames.
  • -u – converts all [U]nderscores to spaces.
  • -s – converts all [S]paces to underscores.
  • -l – converts all filenames to [L]owercase.
  • -p – converts all filenames to u[P]percase.

It’d be nice if I could get all working recusively and offer an option to Capitalise Each Word in the filenames, but this will do for now.


Like most scripting, this entire thing could probably be done in a single line, but I found it useful to learn how to use getopts and such.

Hope someone else finds it useful! Oh, and if you need a do-everything file renaming script, I read that FixNames script does what it says on the tin.

How to: Compile Qt projects which use the Q_OBJECT macro in Code::Blocks

I need to write some C++ cross-platform GUI code soon, and after umming-and-ahhing about whether to use GTK or Qt, I’ve decided to try out the latter despite the recent Microsoft buy-out. However, as soon as I started to get into it and write some simple GUI code, I hit a snag; Qt uses what it calls slots to bind buttons to functions, where a function that can be executed as an action is defined as being a slot. To give an example, if you had a integer counter and you wanted to add five to the counter when you clicked the “Plus Five” button, then your onPlusFive function which actually increments the counter would be considered a slot.

This is all well and good, until you write the code and it won’t compile – instead giving you errors about vtables and such. To fix this, you need to do two things, the first of which is to add the Q_OBJECT macro definition to your QWidget-extending class.

Q_WHATNOW?

I’m really new to Qt, so any experienced coders might roll their eyes at this – but I expected to be able to just write some code that creates a window, some buttons, layouts, menu options etc, and then just compile it and we’re good to go a-la Java’s AWT/Swing interface kits. But no, unfortunately not. Instead, you have to add the Q_OBJECT macro to your class like this, and then do some pre-processing:

That doesn’t seem too bad, right? You just plonk the word Q_OBJECT at the top of your class, and list all slots (functions that get executed as button actions) under the private slots: label. Well, yeah – it’s not so bad, but it’s not going to work that easily either. For this to work properly, we also need to do some pre-processing with a Qt utility called moc

Moc Pre-processing Steps

AFAIK, the pre-processing that needs to occur must happen on all header files (and header files only) that utilise Qt in your project. You can do it by hand like this:

For example, if you had a MainWindow class separated into .h and .cpp files, you might run:

Once that’s done, you need to add the newly generated moc_MainWindow.cpp file (for this example) to your Code::Blocks project, and it’ll all compile and work as expected.

So good so far, but what happens when you add a new slot? Yup, you have to go and manually re-run the command on all header files to generate the new moc_<some-class>.cpp files before it’ll play ball again – well sod that, let’s automate it!

Quick & Dirty Pre-Processing via Bash

I’m sure there are better ways of invoking the moc as a pre-build script, but I don’t know enough about pre-processing scripts to get it working (however, you might like to read this and have more luck than I did) – so I knocked together a quick bash script to generate all the moc files for us, like this:

genQtMoc Script

This is fine if all your .cpp and .h files are in the same folder like the project root, but if you’ve split them off into include and src directories you’ll have to twiddle the script a little to take that into account.

Putting it to work

Once you’ve put the script into a file called genQtMoc or such, and made it executable with chmod +x genQtMoc, sling it somewhere in the path like /usr/local/bin/ and we can then call for it to be executed as a pre-build script in Code::Blocks by going to Project | Build options… | Pre/post build steps and just putting the name of the script to call in the pre section, like this:

Code::Blocks Pre-Build Script

All sorted – the only other thing to re-iterate is that you need to have each moc_<some-file>.cpp file attached to your project, and that if you have the file open for editing in Code::Blocks, it’ll moan that “the file has been changed externally, would you like to reload it?” each time its contents change. You can get around this by simply not having the file open for editing (but still included in your project), which stops the annoying pop-up from occurring and is fine as it’s an auto-generated file that shouldn’t really be hand-edited anyway.

Anyways, I hope this helps anyone stuck in a similar situation – and please, if you do know a better of way of going about this, I’d love to hear about it. Cheers!

How to: Get absolute/relative file paths, filenames and extensions from a Bash script

I’ve been learning some Qt stuff today, and the resulting code requires pre-processing of header files to work correctly, which I thought I’d try to automate. As part of this, I needed to find out how to get at all the different elements of a file from a bash script, so I did some googling and got all the info I needed :)

The script itself isn’t particularly useful, but the component parts of how to get at paths, extensions, and plain filenames without extensions definitely is – check it out:

fileParts Shell Script

I created two dummy “.h” files in my home folder and ran the script – this is what it outputs:

That’s gotta come in useful, right? =D

Help scrape Google Video before it’s gone forever!

Update: Google About-Turn

Google have capitulated to feedback and decided to keep Google Video alive and migrate the videos to YouTube, by which point Archive Team had 40% of the content and were well on track to save it.


Google Video will be shutting down within the next few weeks, and for some stupid reason, they’re not just transferring the videos to YouTube (as Google owns both) – instead they’re just pulling the plug and it’s all going to be lost. To fix this rubbish state of affairs, Archive Team are in a race to scrape as much Google Video content as they can before the viewing deadline (29/04/2011) and the download deadline (13/05/2011) – and you can help! Archive.org have kindly donated 100TB for storage, but first we need to index the videos and scrape them.

If you have a lot of bandwidth you can help scrape the videos themselves, but even if you don’t you can help with the indexing effort by running a simple, resource and bandwidth light Linux script and just leaving it running!

Why save it?

YouTube has a 15 minute video length limit – and Google Video doesn’t. This means there are large amounts of video that might be on Google Video and nowhere else, so when they’re gone – they’re gone. A lot of this might not be fantastic material – but a lot of it will be unique and the only copy on the Net. There’s documentaries, films, and all sorts of good stuff, and even personal video blogs will be a snapshot of the times we live in. In short, it’s stuff we as a species, should not throw away.

It’s like the BBC scrapping their archives when they didn’t want to pay to save them – we won’t know what’s been lost until it’s gone, and by then it’ll be too late. So let’s not let that happen, eh?

How you can help if you have ~200GB bandwidth/storage or more: Scrape the videos

Update: pentium ported the video download script to Windows (you still need python and aria2, which can be downloaded separately). Script location: http://www.pentium100.com/gg_windows.zip

Head on over to ArchtiveTeam Google Videos wiki, sign up, pick an un-taken section of videos and add your initials/handle to it, then go for your life! Full instructions at the site.

How you can help if you don’t have a lot of bandwidth/storage

Update: nstrom ported the related.sh indexing script to Windows – all you now need is a pre-compiled version of the phantomjs browser. Full instructions come with the script, which is located here:
http://nstrom.chaosnet.org/google_video_related_win.zip

Note: This will only work on Linux machines with X running – you can’t run it on headless servers due to phantomjs requirements. Instructions are for Ubuntu 10.10 or later and might need a little modification if you’re running an older or non-Debian based distro.

  1. Get and build phantomjs (a headless web browser) by doing the following:
    • Install build-essential, curl git, libqtwebkit4and libqtwebkit-dev if necessary, for example using:
    • Create a directory called phantomjs
    • In the terminal go into your new directory and run the following command to get the phantomjs source code:
    • Build phantomjs by issuing the command:
    • Move the phantomjs binary somewhere in your path by issuing the command:
  2. Create a folder called gvscript or such and download the file with the list of Google Video related pages to scrape: google_video_related.tar.gz
    • Extract the above downloaded file (Right-click and Extract To.. or use tar -zxvf ./google_video_related.tar.gz)
  3. In a terminal, navigate to the folder where you extracted the google_video_related file (above) and run the following command to help scrape Google Video:
  4. Simply leave the script running, and head on over to #ggtesting on EFnet (IRC) if you need any assistance or in case the script has any issues (p.s. kill the script with Ctrl+Z if it misbehaves – though mine’s been running for about 7 hours solid with no complaints so I doubt you’ll have any).

The script scrapes each page for related videos and sends them off to an archiveteam server. It takes very little processing and bandwidth on your end (a couple of kb/sec, if that) and seems to work just fine.

Every little helps

I’m sure anything you can do to pitch in will be appreciated by Archive Team, the Internets, your future self, your kids, your kids kids, your kids kids kids… you get the picture ;)

Cheers!

How To: Convert a Directory of MP3s to WAVs in Linux

I went to burn a couple of audio CDs for the boy today, and bod-frickn-dammit if Brasero / GnomeBaker and K3B didn’t all threw their hands in the air in dismay that I might actually have the nerve to want to convert mp3s to CD-audio on the fly. Very poor.

So, as wav files come in less flavours than mp3s and tend to work first time, I knocked up a quick script to convert a directory of mp3s to .wav files using mpg123, it even handles spaces correctly after I’d given it a stern talking to… Anyways:

To use the script:
– Copy and paste the above code into a new text file called mp32wav or something
– Make it executable with chmod +x mp32wav, then
– Copy it to /usr/local/bin for easy access with sudo cp mp32wav /usr/local/bin/

With that all done you can just go into a folder of mp3s in the terminal and fire it off. It’ll create a folder called WAV inside whatever directory you’re in and stick the converted wav files there with the original filename but with .wav tacked on the end.

Cheers!