Warning! Geekiness ahead

Ok, so just had a wonderful time trying to set up a way to automatically backup my hostgator VPS to Amazon S3.

I did it with the help of some googling, head scratching and tricky bastardness so I thought I’d better document it here to remind me how to do it when I need to do it again.

I got most of the information from the post here and in the comments. Although, I had to piss about a lot to get it finally working. Mainly because of curly quotes in the original page, windoze line breaks and my monkey heritage.

(this is for VPS on Hostgator running Centos 5)

1. Create AmazonS3 bucket

Easy bit here, just create a bucket

2. Install S3 client for Linux

First need to install s3tools repo

cd /etc/yum.repos.d
wget http://s3tools.org/repo/CentOS_5/s3tools.repo

Next need to install

yum install s3cmd

Answer the questions with Y

3. Configure s3cmd

s3cmd --configure

Enter in the access key and secret key from Amazon Security Credentials

4. Enable daily backups from WHM

You can select which accounts by clicking the button marked ‘select’

If it’s already configured, find out the backup directory by typing

grep BACKUPDIR /etc/cpbackup.conf

5. Create the log directories

mkdir /var/log/backuplogs

6. Write a script to automate the backup and save it as /root/dailybackup.sh

You should change the email and bucket name to reflect your own values

#!/bin/bash

##Notification email address
_EMAIL=youremail@yourdomain.com

ERRORLOG=/var/log/backuplogs/backup.err`date +%F`
ACTIVITYLOG=/var/log/backuplogs/activity.log`date +%F`

##Directory which needs to be backed up
SOURCE=/backup/cpbackup/daily/*.gz

##Name of the backup in bucket
DESTINATION=`date +%F`

##Backup degree
DEGREE=3

#Clear the logs if the script is executed second time
:> ${ERRORLOG}
:> ${ACTIVITYLOG}

##Uploading the daily backup to Amazon s3
/usr/bin/s3cmd -r put ${SOURCE} s3://yourbucketname/${DESTINATION}/ 1>>${ACTIVITYLOG} 2>>${ERRORLOG}
ret2=$?

##Sent email alert
msg="BACKUP NOTIFICATION ALERT FROM `hostname`"

if [ $ret2 -eq 0 ];then
msg1="Amazon s3 Backup Uploaded Successfully"
else
msg1="Amazon s3 Backup Failed!!\n Check ${ERRORLOG} for more details"
fi
echo -e "$msg1"|mail -s "$msg" ${_EMAIL}

#######################
##Deleting backup’s older than DEGREE days
## Delete from both server and amazon
#######################
DELETENAME=$(date --date="${DEGREE} days ago" +%F)

/usr/bin/s3cmd -r --force del s3://yourbucketname/${DELETENAME} 1>>${ACTIVITYLOG} 2>>${ERRORLOG}

7. Grant execute privilege to the script

chmod u+x /root/dailybackup.sh

8. Set up a cpanel hook to run the script after the backup has completed

nano /scripts/postcpbackup

enter this as the contents

#!/usr/bin/perl
system("/root/dailybackup.sh");

make it executable

chmod u+x /scripts/postcpbackup

That’s it!

In case of disaster, copy the file from Amazon s3 with

mkdir restore
s3cmd -r get s3://yourbucketname/2011-02-32/filename.gz restore

Notes
I changed the bash script so it only copies *.gz files
I wanted to change the time at which the cpbackup occurs so I went to ‘manage plugins’ in WHM and put ‘install and keep udpated’ tick on ‘cronconfig’ and then went to ‘configure cpanel cron times’ and set the time I wanted cpbackup to run.

Twice. In 1 week.

I lost my code and my awesome plugin edits when my hard drive crashed, I started writing it again and I formatted the wrong partition. Seriously gutted! I tried to recover the data with undelete apps but it was gone forever.

No worry no more!

Now, I don’t need to worry because I use Dropbox and have configured my local server to use it as the document_root of my dev site. That way, all the developing work I’m doing gets saved automatically online and is accessible on any of my pc’s.

I don’t even need to worry about being able to access the SAMBA share if I want to edit a file, I just open it from my dropbox on the pc I’m using and save it back there, it’ll get automatically updated on my dev server.

Oops no more!

Another advantage is the revisions feature. You can go online and see the updates that you made to a file and revert it back to an earlier save. This has already saved me from crying, I always forget to rename a file before I start trying to add updates and before I know it, the whole thing is causing an error and I can’t remember how to put it back the way it was! just luvin the revision feature!!

Get 2GB for FREE! (and help me out)

You can apply for a Dropbox for FREE and get a full 2GB to use to store anything from simple documents to executable files to database backups or like me, use it as your code repository and never go through the pain of a lost bit of awesome code again!

You can even install it on Linux which is super helpful when it comes to transferring files between a windows->linux network.

If you click this link and install your dropbox for free, we both get extra bonus space which would help me out a lot

I can’t believe what I just did….. I mistakenly restored an old backup of fiddyp… grrrrr.

I have just moved to a dedicated server and its really helped the load times, I got everything working and then did a backup. “sweet” thought I…. let’s set up a scheduled backup… two wrong clicks later and I realise that it’s restoring the old back up. dammnit all to hell on a bicycle!!!!

Luckily, I managed to retrieve the last two posts I made by copying the source of the feed in google feedreader, amazingly, that still held the last two posts I made. Shame I missed the comments though, I’ll have to see if I can find a list of the entries of the Euromillions rollover contest in a history somewhere and get it from offline mode….

silly silly me!

edit: I just realised that I can reconstruct the comments from my email notifications. I just got all the entries back on the euromillions contest post from a page view in the history, i’ll put the remainder up when I get over my heart attack!

newedit: I tried to change this hosting to exclusive IP on a dedicated server and I lost another couple of comments (sorry Sharon). I have had the most frustrating day playing with this server. I feel an indepth rant coming!….

even newer edit: I think I’m changing too many things and not giving it time to work! I am now going to leave the nameservers as they are, leave the IP of the hosting the same and just sit back and wait for a few days for everything to settle. sorry for any interuptions to normal service.. I’m trying to get everything up and secure on a fast host so I can get my new plans in to gear and not worry about bandwidth affecting download times and other schtuff..

seeya soon!