Configuring Hostgator VPS for automated CPanel backups to Amazon S3

Warning! Geekiness ahead

Ok, so just had a wonderful time trying to set up a way to automatically backup my hostgator VPS to Amazon S3.

I did it with the help of some googling, head scratching and tricky bastardness so I thought I’d better document it here to remind me how to do it when I need to do it again.

I got most of the information from the post here and in the comments. Although, I had to piss about a lot to get it finally working. Mainly because of curly quotes in the original page, windoze line breaks and my monkey heritage.

(this is for VPS on Hostgator running Centos 5)

1. Create AmazonS3 bucket

Easy bit here, just create a bucket

2. Install S3 client for Linux

First need to install s3tools repo

cd /etc/yum.repos.d
wget http://s3tools.org/repo/CentOS_5/s3tools.repo

Next need to install

yum install s3cmd

Answer the questions with Y

3. Configure s3cmd

s3cmd --configure

Enter in the access key and secret key from Amazon Security Credentials

4. Enable daily backups from WHM

You can select which accounts by clicking the button marked ‘select’

If it’s already configured, find out the backup directory by typing

grep BACKUPDIR /etc/cpbackup.conf

5. Create the log directories

mkdir /var/log/backuplogs

6. Write a script to automate the backup and save it as /root/dailybackup.sh

You should change the email and bucket name to reflect your own values

#!/bin/bash

##Notification email address
_EMAIL=youremail@yourdomain.com

ERRORLOG=/var/log/backuplogs/backup.err`date +%F`
ACTIVITYLOG=/var/log/backuplogs/activity.log`date +%F`

##Directory which needs to be backed up
SOURCE=/backup/cpbackup/daily/*.gz

##Name of the backup in bucket
DESTINATION=`date +%F`

##Backup degree
DEGREE=3

#Clear the logs if the script is executed second time
:> ${ERRORLOG}
:> ${ACTIVITYLOG}

##Uploading the daily backup to Amazon s3
/usr/bin/s3cmd -r put ${SOURCE} s3://yourbucketname/${DESTINATION}/ 1>>${ACTIVITYLOG} 2>>${ERRORLOG}
ret2=$?

##Sent email alert
msg="BACKUP NOTIFICATION ALERT FROM `hostname`"

if [ $ret2 -eq 0 ];then
msg1="Amazon s3 Backup Uploaded Successfully"
else
msg1="Amazon s3 Backup Failed!!\n Check ${ERRORLOG} for more details"
fi
echo -e "$msg1"|mail -s "$msg" ${_EMAIL}

#######################
##Deleting backup’s older than DEGREE days
## Delete from both server and amazon
#######################
DELETENAME=$(date --date="${DEGREE} days ago" +%F)

/usr/bin/s3cmd -r --force del s3://yourbucketname/${DELETENAME} 1>>${ACTIVITYLOG} 2>>${ERRORLOG}

7. Grant execute privilege to the script

chmod u+x /root/dailybackup.sh

8. Set up a cpanel hook to run the script after the backup has completed

nano /scripts/postcpbackup

enter this as the contents

#!/usr/bin/perl
system("/root/dailybackup.sh");

make it executable

chmod u+x /scripts/postcpbackup

That’s it!

In case of disaster, copy the file from Amazon s3 with

mkdir restore
s3cmd -r get s3://yourbucketname/2011-02-32/filename.gz restore

Notes
I changed the bash script so it only copies *.gz files
I wanted to change the time at which the cpbackup occurs so I went to ‘manage plugins’ in WHM and put ‘install and keep udpated’ tick on ‘cronconfig’ and then went to ‘configure cpanel cron times’ and set the time I wanted cpbackup to run.

36 thoughts on “Configuring Hostgator VPS for automated CPanel backups to Amazon S3

  1. Genie

    I’ve long in search for that an d I finally have it answered. It saves a lot of time and get back up. I completely understand the whole point about it.
    Genie recently posted…skin tagsMy Profile

     
    Reply
  2. Cheolsu

    I recently started using a VPS hosting for one of my blogs. I should try setting up a similar automated backups.
    Btw, I am glad that I stumbled upon your personal blog. I am a fan of your commentluv plugin. Good luck with your commentluv premium launch.
    Cheolsu recently posted…Myspace Login with FacebookMy Profile

     
    Reply
  3. Jun

    Thanks for sharing this information. In the past, I create back up once a week, with this I won’t have problem backing up and its automatic. Also, it’s step by step making it useful for me who have just basic understanding on these kind of stuff.
    Jun recently posted…how to meet womenMy Profile

     
    Reply
  4. Clifford

    Thanks for this. I’m not on HostGator, but all’s been working fine. :)

    How can I delete the backup from the server after 1 day but from S3 after 30 days (or possibly even delete from server immediately after upload completed successfully – would that be 0 days)? Instead of deleting from both sources at the same interval.
    The comments in /root/dailybackup.sh say that the DEGREE variable is used for both server and S3, but the command that you use (/usr/bin/s3cmd -r –force del s3://t…..) looks like it would only delete from S3.

    Thanks!
    Clifford recently posted…Annotate websites and share your edits (Freebies)My Profile

     
    Reply
      1. Cliff

        The code includes this comment:
        #######################
        ##Deleting backup’s older than DEGREE days
        ## Delete from both server and amazon
        #######################

        Yet the code’s only delete/rm command is on S3, not on the local server (as the comment mentions).

        Looking in the /backup/cpbackup/ daily, monthly, etc directories, the backup files remain as …tar.gz files.
        And that makes sense since there’s no ‘rm’ command to the local server, only to S3.

        Are you sure you’re using the same script that you’ve posted here?

        tyvm
        Cliff recently posted…iPhone camera shutter button – POPAMy Profile

         
        Reply
          1. Cliff

            Correct. Same as mine. Thanks.

            Would you mind suggesting a command to explicitly remove the server’s backup after uploading to S3?

            Also, is the /daily .tar.gz file the same one as in the /monthly folder?

            I don’t want a /daily or /monthly or /yearly folder – I just want backups on S3… maybe just 1 backup on local server would be ok.

             
          2. Clifford P

            How can i get this to happen at the end of the script?

            ##Remove local backup folder after S3 PUT
            rm -rf /backup/cpbackup/monthly
            rm -rf /backup/cpbackup/weekly

            I appreciate the assistance.

             
    1. Andy

      haha actually I had problems with backbuddy and so I used this because it runs independently of wordpress and doesn’t cause wordpress to bork if another plugin is using too much memory
      Andy recently posted…CommentLuv PremiumMy Profile

       
      Reply
  5. Paul

    Thanks for the post, really looks like a great idea. I got the folloing erro though during setup:
    yum install s3cmd
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
    * base: mirror.raystedman.net
    * extras: mirror.raystedman.net
    * update: mirror.raystedman.net
    Segmentation fault (core dumped)

    What is the matter? I’ve tried looking into this but cant see any specific…

     
    Reply
      1. Paul

        Thanks for the heads up, got it sorted.

        There was an issue with libxml which needed to be recompiled. To solve it I had recompile libxml and updated YUM.

        lets hope it works :)

         
        Reply
  6. Jesus Perez

    I just set this up on my WiredTree VPS using the http://s3tools.org/repo/RHEL_6/s3tools.repo since I’m running CentOS_6. The instructions were smooth as silk. Thanks so much, Andy. Only one minor step is missing. CTRL+X to exit Nano and “Y” to save.

    Question: when I copied the backup.sh code, I see some very indented single quotes. Are these ok? Or should I switch them to the the straighter single quotes in my text editor?
    Jesus Perez recently posted…Why I Love WiredTree HostingMy Profile

     
    Reply
    1. Andy

      glad you got it working for your site Jesus. the indents will probably by a copy and paste thing, you should be ok to use the quotes that you are used to using but if it works, keep it as is or keep a backup copy in case the changed quotes don’t work on your system
      Andy recently posted…CommentLuv PremiumMy Profile

       
      Reply
  7. Jesus Perez

    Thanks Andy!

    One last tip. To force the Cpanel backups and immediately test the setup, use the following commands in SSH:

    For newer VPS servers using the newer backup system:
    /usr/local/cpanel/bin/backup –force

    For older VPS servers using Legacy backup system:
    /scripts/cpbackup –force

    Just keep in mind it can 1) take a while depending on your size and 2) it can bring your server down while it’s working.
    Jesus Perez recently posted…Why I Love WiredTree HostingMy Profile

     
    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

CommentLuv badge