How To: Set up a FTP Server in Linux

A mate of mine who shall remain nameless [cough]Shetboy[/cough] wanted to send me a 30MB file the other day, so at 30MB it’s not going to fit through email without splitting the file up and sending multiple emails with a section-per-mail, and we didn’t want to use dropsend either, so I ended up spending the best part of a day researching and setting up a FTP server on my box so he could just connect and upload the file. Here’s how I did it:

1.) Get yourself a FTP server

pro-FTPd seems to be the server of choice when I was researching, so I went with that and a copy of the docs (note that I’m not going to be using mysql for the user authentication, so I don’t need to get proftpd-mysql as well):

When installing, it’ll pop up a window asking if you want a Standalone installation of FTPD, I did, and you probably do too, so just hit Next | Next | Finish or whatever to complete the installation.

2.) Get yourself a GUI to configure the server

This is optional, and you might just want to get elbow-deep in the config files, but I decided to use a GUI to configure most things, and then just tweak the resulting config file:

With that done (in Gnome, at least) you’ll have a set of GUI tools for configuring things, including proFTPd in your Applications | System Tools menu.

3.) Configure the FTP Server

This is the fun bit…

– Open GADMIN-PROFTPD and it’ll moan it doesn’t have a valid configuration file and ask you if you’d like to create a standard installation configuration, why yes, yes you would.
– As I’m not using mysql for authentication, and I don’t want to have to use real users of my system to connect, I’m going to use virtual users. To do so, just check the Enable virtual users checkbox.
– I don’t want my FTP running on the standard port (21), so I’ve moved the ftp service to run on port 2121
– I’m behind a router, so I need to tell proFTPd what my external IP is, for this guide, lets just say it’s 1.2.3.4. Enter whatever your external IP is (whatismyip.com comes in useful here) and don’t forget to change the NAT Router setting to On to enable it.
– We’re going to be using additional ports (i.e. passive FTP), so we need to leave the range of 49152 to 65534 alone, and we’re going to port-forward these from our router to our internal server IP later.

You should end up with something looking pretty much like this (bar entering YOUR external IP, not just 1.2.3.4):

proFTPConfig1proFTPConfig2

4.) Set up your Virtual Users and Groups

If we want to use actual system users (i.e. user accounts which exist to log in to our machine) we can, but it’s a bit safer to use virtual users, and do you really want to create additional accounts on your machine when you don’t have to? Thought not. So to create our virtual users we use a script called ftpasswd which comes with proFTPd.

From the shell enter the following:

Explanation:

  • The ftpasswd tool creates your user and group files in your present working directory, if we move to where we want them to be created first, we don’t have to move them later.
  • The name field is the username you want to use to log in to your ftp server.
  • /home/ftpuploads is a directory I’ve created to store all the FTP files we’re going to serve, and is the location files will be uploaded to. You’ll need to create this directory, or another directory and then set them to be owned by the nobody group and user by using sudo chown nobody /home/ftpuploads and sudo chgrp nobody /etc/ftpuploads commands.
  • –shell=/dev/null disables any FTP users from starting their own shell session where they could potentially cause mischief.
  • –uid=xxxx and –gid=xxxx – A virtual user must have a UserID and GroupID. You don’t have to use 5000 for them, just any unused User and Group IDs will do – and I picked 5000 out of the air as they’re both free on my box.

After hitting enter you’ll be prompted for the user’s desired password and confirmation of the password, then a file called ftpd.passwd will be written in your current directory.

Next we need to create the group access file using ftpasswd again as follows:

The group name can be anything you want, and make sure you put the same gid value as you used when creating your user(s) above. If you’ve got multiple users you want to be able to connect to your FTP server, just add on additional –member=someuser –member=anotheruser switches when creating the group.

If you want to add additional users later on, just create a new user using ftpasswd as we’ve done above but specify a different, unique UID for the user, then edit the ftpd.group file and tack on the users you’ve just created. For example, you’d modify the ftpd.group file so that:

became:

Note: You may need to add write permissions to ftpd.passwd and ftpd.conf via sudo chmod 644 ./ftpd.passwd and sudo chmod 644 ./ftpd.group if you want to manually edit them.

Finally, don’t forget to add in your new users as allowed to login to the FTP server from the proftpd.conf file by finding the section at the bottom that looks like (angle brackets changed to square brackets in the below text to avoid HTML woes):

and changing it to something like:

5.) Tweak the config

We used GADMIN-PROFTPD to create our initial configuration (which is stored in /etc/proftpd/proftpd.conf), now we’re going to tweak it a little to use our created passwd and group files. To do this, add in the following lines to the top of your proftpd.conf file (making sure you don’t have the same directives twice! i.e. if there’s another AuthUserFile directive somewhere, remove it etc.):

Once you’ve done this, strip out the Anonymous section from the bottom of the file – we don’t want anonymous logins, and GADMIN-PROFTPD uses a cludgy, incorrect way to try to add in users as anonymous users anyway.

You should end up with a file looking something like the one here.

6.) Test it locally

Stop and then start the FTP server so it’s running with our new config by using:

Try connecting to your server in Firefox on ftp://localhost:2121 – if you’ve followed the steps correctly you should be able to log in. If not, have a look to see if you’ve missed any steps, and maybe do a little additional reading on Virtual Users, Authentication, and basicconfig.com has a proFTPd setup guide which I found halfway through writing this one!

7.) Open it up to the world

If you can connect from inside your local network, and you want users to be able to access the FTP server from anywhere, we need to do a bit of port-forwarding to allow the server to be accessed from the outside world. Port forwarding is a big topic, and I’m not going to cover it in depth here – if you need help try portforward.com, but what we’re going to do is forward port 2121 to the local IP of our FTP server (in my case the FTP server’s just running on my laptop at local IP 192.168.1.101), and then port forward the range of ports 49152 to 65534 for our actual FTP traffic.

I tried doing this initially with the DD-WRT firmware on my router, but the router config had got a bit mangled and wouldn’t work properly. As I’d been meaning to move to the Tomato firmware anyway, I just threw that on the router and things started working properly in no-time, so just be aware that your router and not your FTP configuration can be the weak link when things aren’t working sometimes.

proFTPConfig3

With that done, try accessing your FTP server through a web browser with: ftp://YOUR-EXTERNAL-IP:2121

If you get a login box then your port-forwards are fine. If you can’t login you need to stop the FTP server and go look at the Virtual Users part again then restart the FTP server. If you can access and login, but can’t upload or download, you need to look at your ftpuploads directory and maybe change the permissions on them (try sudo chgrp 5000 /home/ftpuploads -R, for example).

Last thing, try using FileZilla or something to upload some files to your FTP server – if it works, you’ve done it. Congrats!

Oh, and the mate who wanted to FTP me a 30MB file? He’d misread the file size – it was actually 3MB… ls -alh ftw!

Final note: If you’re running DD-WRT on your router, and want to change to the Tomato firmware, don’t forget to telnet into the router and get the password hash first!

How To: Fix (Workaround, really) VirtualBox High CPU Load with Ubuntu 9.10 Host

Updated to Karmic the other day, and all is well and good with the world, but VirtualBox is determined to use a heap of CPU time even when the guest OS is idling. This sucks, and from what I’ve read is related to CPU affinity.

I tried changing the affinity from 3 down to 1 with sudo taskset -p 1 31540 from the terminal, where 31540 was the PID (Process ID) of VirtualBox at the time – and did it work? Um, no…

After some further reading I found a quirky, but totally working fix workaround:
– Create a new virtual machine, call it “Dummy” or something
– Assign it as little resources as you can, like 4MB RAM, use a 10MB HD image etc
– Start this dummy virtual machine, skipping any virtual hard drive formatting
– That’s it! Just leave the dummy machine running at the screen saying “FATAL: No bootable medium found! System halted”

With the dummy machine running, VirtualBox sorts out its own processor affinity properly, and the CPU usage of your actual VM guest drops waaaaaaay down and becomes snappy & responsive again.

The joyz…

Update: This problem’s been recurring for me (as you can see in the comments below), but I think I’ve cracked it: With an XP guest running on a single core of a Core 2 Duo, if I run the following command in bash I can get the CPU usage down to negligible levels:

If this works for you and you want it to be applied at every boot, just edit the file /etc/sysctl.conf and add the line kernel.perf_counter_paranoid = 2 at the bottom.

Additional Information: Although perf_counter_paranoid 2 seems to work reasonably well – we can still do a bit better. With an XP guest, once you’ve set XP to use multiple processors, it’s going to keep trying to use multiple processors, even if you’ve told VB to only allow it access to a single CPU. To fix this you can grab yourself a copy of of HALu and change your XP CPU setup from ACPI Multiprocessor PC to ACPI Uniprocessor PC. With that in place, CPU usage for your linux host drops down a little further, and doesn’t ramp back up too far under minimal guest load. Or you could just use the dummy VM… FFFFFFUUUUUUUUUUUUUUUUUUU…. =P

How To: Fix Stuttering Sound in 9.04 Jaunty Jackalope

Just upgraded my 8.10 Intrepid Ibex Ubuntu distro to 9.04 Jaunty Jackalope, and bar a slight keyboard configuration issue (paraphrased as: “current layout not found – will leave keyboard config alone”) and having to take a close look at my GRUB menu.lst before deciding to take the package maintainers version (new ver includes updated entries of your current ver – but backup your current menu.lst just in case!) everything went fine. In fact, 9.04 feels more fluid & responsive in some aspects, so all good so far.

The only problem I’ve noticed (and fixed) so far is that using my external Creative soundcard, and likely the Intel onboard card, sound stutters a bit. This is due to ALSA’s “glitch-free” (I kid you not) drivers, having, er, glitches when used through PulseAudio. I guess you could remove PulseAudio, if you really wanted to, but there’s a simple one line fix – just edit the file /etc/pulse/default.pa and add following line:

Then, either restart PulseAudio with /etc/init.d/pulseaudio restart or reboot – and job done – no more glitching sound.

A quick gnome-based sound test is to run the Sound application in System | Preferences (i.e. gnome-sound-properties) and just click the [Test] button on Sound Events | Sound Playback.

From some further reading, it seems tsched=0 is a kludgy workaround that can cause higher CPU usage for sound playback, and the real problem lies with the Ubuntu kernel being high latency.. (see Ubuntu Forum link below, post #43 onwards). I think I’d rather higher CPU usage than the sound breaking up on me, and playing some mp3s in VLC (just because the new Amarok’s still busy scraping together collection details from the NAS :) ) takes 2% of a single core on my laptop. When running at the lowest possible speed of 800Mhz.

I don’t think that’s gonna be a problem…

Sources: lglinux, ubuntu forums

Update:
Even with the above fix, sound would sometimes be a pain on an upgraded 9.04 – mute channels you had to unmute in alsamixer, xine and Gstreamer engine config woes, mplayer has sound but vlc doesn’t, or vice versa, or neither have sound but firefox does… I decided to just wipe the entire system (backing up the /home partition first for a file-system change over) and start again clean.

I think the glitches were from upgrading 8.04 to 8.10 to 9.04 and everything being a mish-mash of legacy code and deprecated configs held together with gaffer-tape and bubblegum… It wasn’t pretty. But with a fresh system slapped on EXT4 partitions, I get zero sound issues, the system boots and runs quicker than I’ve ever seen it go, and it only took a little while of checking some boxes in synaptic to get things back to pretty much where I left off. I’d definitely recommend installing 9.04 fresh – nothing else has that minty new-OS zing, or lack of seriously annoying glitches. Final Note: Be aware that if you go for EXT4 as your filesystem you will have to set some options and cross your fingers if you want to resize the partitions using the tools available in Jaunty, and that there can be a problem with delayed allocation and 0-byte files if the box falls over before committing data. If that doesn’t sound like it’s for you, XFS is fast and safe – and knocks EXT3 into a tinfoil-hat.

Update 2:
I somehow managed to get it so Nautilus and Firefox would play sound (through PulseAudio), but VLC or MoviePlayer or anything else wouldn’t… no idea how – perhaps by having the audacity to use my frackn machine? So obviously some programs are using pulseaudio, which seems to work, and others are trying to use ALSA, which isn’t working because pulseaudio is raping it… Anyway, I tried about 5 things to fix the sound – here’s some details:

1.) From the Simple Guide to Sound on Hardy, Intrepid & Jaunty, I ran:

and rebooted. Did this help? No. And most of it was installed already.

2.) I upgraded ALSA to 1.0.19 (while 1.0.18 is the one in official jaunty repos at the mo) using the script here. Did this help? Not immediately. But it won’t have hurt.

3.) I installed vlc-plugin-pulse – and after changing the audio output to Pulseaudio server, VLC would produce stuttery sound, which is a start.

3.) I read a bunch of this stuff: Multiple Sound Solution, some more, and then even more.

4.) I went System | Preferences | Default Sound Card and chose MY SOUND CARD – *not* pulse audio (you can also do this from the command line with: asoundconf set-default-card CARDNAME – to find out which cards are available, run: asoundconf list). I then went to System | Preferences | Sound and changed all my settings away FROM Pulseaudio TO Alsa Mixer for my soundcard (the reason I say my soundcard is that I’ve got an onboard Intel soundcard, and an external USB Creative one I prefer to use because it has optical input/output) – if you hit the [Test] button with ALSA used for playback and it doesn’t play, that’s your glitch.

5.) ALSA decided it would play, I changed VLC back to ALSA output from Tools | Preferences | Audio – and *bang* – no more suck-ass, stuttery, crackly pulseaudio sound. Pure clean audio from all applications.

I wish I could be more specific about exactly what fixed it for me when I was having no sound – but I genuinely don’t know exactly what combination of steps fixed things. One minute ALSA mixer wouldn’t play, the next it would – it’s some kind of pulseaudio/alsa conflict, and using ALSA gives me the best sound, when Pulseaudio doesn’t hijack it… There are steps to remove pulseaudio, and at the present time, as much as I like the goal of it it’s not doing the job, so the next conflict I get I’m going to go the full hog and purge.

As a last resort if you have no sound make sure none of your mixers are muted by running alsamixer -c 0 for your first soundcard, alsamixer -c 1 for your second etc.. and make sure none of the playback mixers have MM (i.e. Muted) on them at the bottom, pressing ‘M’ on them will unmute and change it to the bizarely named ’00’ – to do this from a nicer GUI, just install gnome-alsamixer.

Last Chance Saloon Update:
If you’ve got sound in some things but not all, check your gstreamer-properties (by typing that very thing at the console) and make sure you’ve got your audiosinks right. gstreamer-properties is just a front-end for the gstreamer part of gconf-editor btw. I’ve discussed it a bit here.

Last LAST Chance Saloon Update on 16/02/2010: If you’re sure it’s pulseaudio which is messing your application up, launch it without going through pulseaudio via pasuspender NAME-OF-YOUR-APPLICATION. I came up against this when trying to fix ScummVM sound the other day under Karmic 9.10.

Good luck!