Backing up is easy with Rsync

21 08 2009

“Always backup regularly!” is a bit of a mantra in the online linux community. You can see where they are coming from though. How many problems have you read about or posted on forums which could have been solved or avoided had there been a recent backup made? Lots.

Although they always bash you for not making regular backups, they rarely tell you how to make the backups.

As daunting as backing up sounds, it really isn’t hard when you have the tools. One of these tools is Rsync. Rsync is an amazing backup utility. Its flexible, open source, and is really reliable. It can be used for local backup – backing up a local file or directory, to another local directory or mounted filesystem, or it can be used for remote backup – backing up a local file to a remote server, or a remote file to a local directory.

In this article, I’ll be focusing on remote backups, using rsync to backup important files and directories to a remote server over SSH. However, with a little tinkering, you can easily mold this into local backups, if that takes your fancy.

0. Install rsync

You’ll want to install this on both client machine and server. This allows you to take advantage of rsync’s two backup methods – “push” and “pull“. “Push” sends a local file or directory to a remote server, whereas “pull” grabs a remote file or directory, and stores it locally, all of which is done over SSH.

1. Configure

While rsync itself needs no form of configuration, we need to be able to SSH into our remote machine, preferably without a password. If you haven’t already set up SSH, you should check out OpenSSH. There are many guide around the vast stretches of the interwebs. I may even publish a guide myself on here some day.

It’s fairly simple to set up passwordless login of SSH. Simply generate a set of SSH keys on the client-side, then give them a blank password. From here, you send the or file over SSH to your server, using the scp tool, or even ftp if your old school. After this, SSH into your remote machine, add the contents of the .pub file to your “authorized_keys” file in the ~/.ssh directory (if it doesn’t exist, create it using “mkdir ~/.ssh/” and “touch authorized_keys”). (To add the contents of the .pub file, simply “cat id_rsa >> authorized_keys“. Then we simply configure the SSHd confuration file to allow SSH access via key authentication.

2. Backing up

Now that everything is in place, we can start backing up important documents and directories!

NOTE: Rsync won’t create a destination directory if one doesn’t exist. You must mkdir your destination directory before using rsync.

If you are using the “push” method (my preference), all you need to do is run this command. As always, I’ll show you the command, and then break it down, making it easier to understand.

rsync –progress -avz -e ‘ssh -p[port]’ /directory/or/file/to/backup user@serverIP:/directory/to/backup/to/

Rsyncis rather obvious. “–progressgive you a progress report along the way. -avzis a mixture of option for how the data is to be handled. astands for archive – this means that permissions and ownerships and whatnot is all preserved. This also means that the directory is sent recursively, ie all the files in all the subdirectories are also sent, with the directory tree preserved. -vstands for verbose – this just increases the output of the command, depending on how many v’s are present (useful for debugging a problem). ztells rsync to compress the data before sending. This speeds up the backup. [port] should be replace with your SSH port. User should be replaced to your main SSH user on the remote host, ie the user you want to use to backup the files. ServerIP should obviously be replaced with the IP address of your server.

For the “pull” method, simply insert your source directory/file as you would with SSH access. ie

rsync –progress -avz -e ‘ssh -p[port]’ user@IP:/file/to/backup /local/directory/to/backup/to/

NOTE: The existence of a slash at the end of the source directory (the first specified directory or file) is significant to what rsync does. It a / exists, then rsync backs up the CONTENTS of that directory. However if the slash does not exist, then the folder itself is sent.

There we have it. A little guide on how to back up remotely. I hope this helped somewhat.


WPA + Linux = Not as much bother as expected

17 08 2009

So we’ve all heard about the weaknesses of WEP. If not, a quick google search should bring up thousands of sites describing WEP’s weaknesses. There are even videos on youtube describing how to compromise WEP security, many using the popular pen-testing tool Backtrack 3, or Backtrack 4 Beta.

With WPA, the problem of a static key is fixed, by WPA changing the key at a packet transmitted/received frequency. If you want to find out more, you can check this wiki article on WPA.

After messing a little with Backtrack 3, I realised how easy it was to crack WEP keys – from booting Backtrack 3 for the first time to finding my WEP key, it took about half an hour, much shorter if I hadn’t had to follow a  guide.

So WPA, or WPA2 is the way forward. Here is a quick description of what I had to do to get my Arch Linux laptop and server connected to my BTHomeHub2.

NOTE: Remember that I’m using Arch Linux, an independent distribution. This means that it uses its own package management system, “pacman”, as well as handling daemons in a different folder. If you’re not using Arch, remember to swap out the distro-dependent commands for commands suited to your distro.

0. Install wireless drivers.

1. Install wpa_supplicant

For Arch Linux:

sudo pacman -Sy wpa_supplicant

After the initial install, running “wpa_supplicant” (without the quotes) will give you a list of supported drivers, including the generic wireless driver WEXT, NDISWRAPPER support and MadWifi, amongst others.

2. Configuring WPA supplicant

I like to backup the default configuration file. However, for this, we’re going to create our own configuration file, so “mv” is used, instead of “cp”. To do this, open your terminal of choice and type:

mv /etc/wpa_supplicant.conf /etc/wpa_supplicant.conf.orig

Now we create the configuration file

touch /etc/wpa_supplicant.conf

Wpa supplicant requires that your SSID and passphrase be encoded into a hexadecimal string. This might sound daunting, but it’s simple if you use a tool bundled with wpa supplicant. Again, in your terminal of choice, run:

wpa_passphrase ssid passphrase

This will output a configuration file that should work from the off with your setup. To save you from typing this out in a text editor and saving it in /etc/wpa_supplicant.conf, we can simply retype that command, but pipe the output to the configuration file.

wpa_passphrase ssid passphrase > /etc/wpa_supplicant.conf

Remember to replace “ssid” with your wireless access point’s name, eg “BTHomeHub2-GKJP”, and to replace “passphrase” with your passphrase.

If your amongst the security conscience of us, you should think about changing the permission of the configuration file, since your passphrase will be stored in plain text. To do this so that only root can read from and write to the file:

chmod 0600 /etc/wpa_supplicant.conf

There we have it, the configuration file for wpa supplicant. Now for connecting to the your access point (AP).

3. Connecting

Before discovering tools such as wicd, I had to connect using wpa supplicant from the command line. Ill detail this way instead of the wicd method, in case your having to connect wireless before you have X installed, as I had to do.

With everything in place, connecting is quite easy. For this, I’m going to use “wlan0” as my wireless device name. However, this name may be something different for you. To find out your device name, run the command “iwconfig”.

First, you must bring your wireless device up. To do this, run:

ifconfig wlan0 up

Now we need to associate with your access point. To do this, simply run:

iwconfig wlan0 essid ssidname

NOTE: Here, you need to replace ssidname with the name of your AP, but leave essid as essid.

Now to connect:

wpa_supplicant -B -Dwext -i wlan0 -c /etc/wpa_supplicant.conf

Since it’s quite a long command, I’ll break it down a bit. The “-B” tell wpa supplicant to run in the background. The “-D” selects the driver to use. The “-i” tell wpa supplicant that your going to be specifying your interface. “wlan0” is the name of your interface (remember to change to your interfaces name). “-c” tells wpa supplicant that next you will be specifying the configuration file’s name.

Nearly done now. All thats left is to ask for an internal IP address. I use dynamic addresses, so to request a dynamic IP, run:

dhcpcd wlan0

And there you have it. You should now be connected to the internet.

One last note for those of us who like to automate long processes like this. You should think about using the “sleep” command between connecting and requesting an IP. I used “sleep 3”, meaning that it waits 3 seconds before requesting an IP address. Some may think this is overkill, but I like to be on the safe side. If you dont “sleep”, then you may run into problems when connecting.

First post!

17 08 2009

Best thing about a blog: you’re guaranteed first post 🙂

So this is Knee Deep In Nix. A humble blog, depicting someones experiences with Linux. It’s nothing special. It’s really just because,

1) I’ve never done a blog before.

2) I might forget the stuff I learn. This is a way to archive it. And,

3) Hopefully it will help other people. Hopefully..

I have this problem you see; I’m never really comfortable with my setup, computer-wise. I’m always googling for ways to make it faster, more efficient, easier, less resource-intensive. Always. I’m sure some of you readers are the same.

I started using Windows XP. I loved it. It was nippy, especially when I saw how sluggish my friends’ computers were due to Vista on pre-vista hardware. I kept the laptop spotless; defragging every few nights, virus-scanning every other night. Not a single problem with it. Except, it always felt kind of wrong – it felt too glossy, asthough a thin plastic film was between me and the software, blocking me a little. As much as I stripped XP back, I could never get passed this feeling. Thats when I found Linux.

I started with Ubuntu. Made a small partition, installed it, along with GRUB – allowing me to dual boot XP and Ubuntu. It was amazing. Until I ran into driver problems with the BCM43xx wireless card. After getting past those, I ran into wireless problems – Ubuntu 9.04 couldn’t connect to my WPA-secure access point (AP).

Then I found Arch Linux. It was a damn big jump, going from the user-friendly Ubuntu to the gritty Arch. But it was great. I got to build a setup, for me. Not for the general consumer, but for me. After again getting past driver support problems, I could install what I wanted, when I wanted, and configure it to exactly how I wanted it.

I started dabbling in Ubuntu around June. I had a full Arch Linux system up and running by the end of July.

Since then, I’ve installed Arch onto a friends old old old PC, and turned it into a fully functional headless file and media server. It’s surprisingly useful.

Anyways, that’s the first post done. Not very exciting, but I thought you deserved some background information :).

What I’m currently listening to:

Jose Gonzalez – Veneer. Entire album on repeat 🙂