Robust backups

Tuesday, December 15th, 2009 at 09:06pm

In light of the failure experienced by two prominent technical bloggers I am glad that over the past few weeks I have been gradually improving my situation in regard to backups by finally crossing a few items off my Todo list.

So what have I done?

Firstly I now have a daily backup of my hosted sites (this one, plus those of some friends). Although I assume that Dreamhost have some form of backup and redundancy, Phil and Jeff learned the hard way that you can’t necessarily trust your host. So in addition to a daily rsync that pulls down all of the hosted files (mostly wordpress files, but also any new images) I now have a server side cron job that dumps each mysql database to a date based file. These mysql dumps are included in the rsync.

It was only last night that I tested these files. From the starting point of a generic Apache with an empty htdocs and an empty mysql database, I was able to copy in the files and import the database. It all worked.

However this is only bringing the backup of my site onto a system that I control. What about the backups of that system?

This is where a pair of external USB drives comes in. The plan with these is to alternate every couple of weeks (at the most) these between home and work. What I have been working on is an automated method to get the data onto one of these drives when it is connected to my windows desktop.

Why the windows desktop?

Because the bulk of what I am backing is 110GB of photos. While these are incrementally synced to my linux box, it is faster to sync them straight from the source. But this is causing some issues with backing up the linux data.

My mail is stored in Maildir format, but when that is copied over windows doesn’t like the file names so they get garbled. So technically I should still have the message content, but I wasn’t sure. So instead I am going to create some archives (tar.gz or possibly rar so I don’t end up with gigabyte sized files) that are then copied over the network.

As this is still a work in progress I expect that the details will change.

Tagged with: