Backing up is easy with Rsync

21 08 2009

“Always backup regularly!” is a bit of a mantra in the online linux community. You can see where they are coming from though. How many problems have you read about or posted on forums which could have been solved or avoided had there been a recent backup made? Lots.

Although they always bash you for not making regular backups, they rarely tell you how to make the backups.

As daunting as backing up sounds, it really isn’t hard when you have the tools. One of these tools is Rsync. Rsync is an amazing backup utility. Its flexible, open source, and is really reliable. It can be used for local backup – backing up a local file or directory, to another local directory or mounted filesystem, or it can be used for remote backup – backing up a local file to a remote server, or a remote file to a local directory.

In this article, I’ll be focusing on remote backups, using rsync to backup important files and directories to a remote server over SSH. However, with a little tinkering, you can easily mold this into local backups, if that takes your fancy.

0. Install rsync

You’ll want to install this on both client machine and server. This allows you to take advantage of rsync’s two backup methods – “push” and “pull“. “Push” sends a local file or directory to a remote server, whereas “pull” grabs a remote file or directory, and stores it locally, all of which is done over SSH.

1. Configure

While rsync itself needs no form of configuration, we need to be able to SSH into our remote machine, preferably without a password. If you haven’t already set up SSH, you should check out OpenSSH. There are many guide around the vast stretches of the interwebs. I may even publish a guide myself on here some day.

It’s fairly simple to set up passwordless login of SSH. Simply generate a set of SSH keys on the client-side, then give them a blank password. From here, you send the id_rsa.pub or id_dsa.pub file over SSH to your server, using the scp tool, or even ftp if your old school. After this, SSH into your remote machine, add the contents of the .pub file to your “authorized_keys” file in the ~/.ssh directory (if it doesn’t exist, create it using “mkdir ~/.ssh/” and “touch authorized_keys”). (To add the contents of the .pub file, simply “cat id_rsa >> authorized_keys“. Then we simply configure the SSHd confuration file to allow SSH access via key authentication.

2. Backing up

Now that everything is in place, we can start backing up important documents and directories!

NOTE: Rsync won’t create a destination directory if one doesn’t exist. You must mkdir your destination directory before using rsync.

If you are using the “push” method (my preference), all you need to do is run this command. As always, I’ll show you the command, and then break it down, making it easier to understand.

rsync –progress -avz -e ‘ssh -p[port]’ /directory/or/file/to/backup user@serverIP:/directory/to/backup/to/

Rsyncis rather obvious. “–progressgive you a progress report along the way. -avzis a mixture of option for how the data is to be handled. astands for archive – this means that permissions and ownerships and whatnot is all preserved. This also means that the directory is sent recursively, ie all the files in all the subdirectories are also sent, with the directory tree preserved. -vstands for verbose – this just increases the output of the command, depending on how many v’s are present (useful for debugging a problem). ztells rsync to compress the data before sending. This speeds up the backup. [port] should be replace with your SSH port. User should be replaced to your main SSH user on the remote host, ie the user you want to use to backup the files. ServerIP should obviously be replaced with the IP address of your server.

For the “pull” method, simply insert your source directory/file as you would with SSH access. ie

rsync –progress -avz -e ‘ssh -p[port]’ user@IP:/file/to/backup /local/directory/to/backup/to/

NOTE: The existence of a slash at the end of the source directory (the first specified directory or file) is significant to what rsync does. It a / exists, then rsync backs up the CONTENTS of that directory. However if the slash does not exist, then the folder itself is sent.

There we have it. A little guide on how to back up remotely. I hope this helped somewhat.





First post!

17 08 2009

Best thing about a blog: you’re guaranteed first post 🙂

So this is Knee Deep In Nix. A humble blog, depicting someones experiences with Linux. It’s nothing special. It’s really just because,

1) I’ve never done a blog before.

2) I might forget the stuff I learn. This is a way to archive it. And,

3) Hopefully it will help other people. Hopefully..

I have this problem you see; I’m never really comfortable with my setup, computer-wise. I’m always googling for ways to make it faster, more efficient, easier, less resource-intensive. Always. I’m sure some of you readers are the same.

I started using Windows XP. I loved it. It was nippy, especially when I saw how sluggish my friends’ computers were due to Vista on pre-vista hardware. I kept the laptop spotless; defragging every few nights, virus-scanning every other night. Not a single problem with it. Except, it always felt kind of wrong – it felt too glossy, asthough a thin plastic film was between me and the software, blocking me a little. As much as I stripped XP back, I could never get passed this feeling. Thats when I found Linux.

I started with Ubuntu. Made a small partition, installed it, along with GRUB – allowing me to dual boot XP and Ubuntu. It was amazing. Until I ran into driver problems with the BCM43xx wireless card. After getting past those, I ran into wireless problems – Ubuntu 9.04 couldn’t connect to my WPA-secure access point (AP).

Then I found Arch Linux. It was a damn big jump, going from the user-friendly Ubuntu to the gritty Arch. But it was great. I got to build a setup, for me. Not for the general consumer, but for me. After again getting past driver support problems, I could install what I wanted, when I wanted, and configure it to exactly how I wanted it.

I started dabbling in Ubuntu around June. I had a full Arch Linux system up and running by the end of July.

Since then, I’ve installed Arch onto a friends old old old PC, and turned it into a fully functional headless file and media server. It’s surprisingly useful.

Anyways, that’s the first post done. Not very exciting, but I thought you deserved some background information :).

What I’m currently listening to:

Jose Gonzalez – Veneer. Entire album on repeat 🙂