Amazon AWS
From RdiffBackupWiki
Contents |
Running rdiff-backup on Amazon Web Services
Amazon Web Services (AWS) are a collection of remote computing services (also called web services) offered over the Internet by Amazon.com. Of particular interest to rdiff-backup users is the Elastic Computing Cloud (EC2) and the Elastic Block Store (EBS).
EC2 is a commercial web service that allows customers to rent virtual computers on which to run their own computer applications, like an rdiff-backup server. Users create virtual machine images which can be launched when needed for an hourly fee. Each time a virtual machine is launched, it is created fresh from a static image.
EBS provides persistent storage for EC2 machines. Volumes can be created and destroyed as required and are charged per gigabyte per month.
Using AWS, you can run a remote rdiff-backup server with flexible persistent storage for off-site backups.
Setting Up Amazon AWS To Run rdiff-backup
Create an AWS Account
First, create an Amazon EC2 account at http://aws.amazon.com/ec2/
Install EC2 Tools
Install and setup the EC2 command line tools on your local machine that is going to be backed up (requires Java) as described at http://docs.amazonwebservices.com/AWSEC2/2008-05-05/GettingStartedGuide/setting-up-your-tools.html
Create an Authentication Key
Create a key pair to authenticate your account by running the following commands on your local machine.
ec2-add-keypair rdiffbackup > rdiffbackup.pem chmod 600 rdiffbackup.pem
This key will be used to authenticate all SSH connections with your virtual machines.
Create Storage Volumes
Create EBS Volumes for permanent storage of your backups by running the following commands on your local machine.
ec2addvol -s <size-in-Gb> -z us-east-1a
us-east-1a is the zone the volume will be created in. Amazon provides a number of zones in both the US and Europe. Zones are isolated from each other for fault tolerance, so the volume must be created in the same zone in which the server will be run.
The command will return a volume-id which uniquely identifies this volume.
Before using this volume you will need to partition and jamu postnatal massage format it. See below about how to initialise the volume.
Create a Security Group
AWS runs servers in predefined security groups. A security group defines what connections, internal and external, are permitted to your server - just like a firewall.
A security group is created by running the following command on your local machine:
ec2addgrp <group-name> -d <group-description>
where group-name is a meaningful name of your choice.
To authorise access to your virtual server, run the following commands on your local machine:
ec2auth <group-name> -p 22 -s <your.internet.ip.address>/24 ec2auth <group-name> -p 80 -s <your.internet.ip.address>/24 ec2auth <group-name> -p 443 -s <your.internet.ip.address>/24
These commands authorise ssh (port 22), http (port 80) and https (port 443) to access your server from your external internet IP address.
Create an Amazon Machine Image
An Amazon Machine Image (AMI) is a hard disk image that is used to launch new virtual machines. The image is copied to a virtual disk and booted to start the virtual machine. Any changes made to the virtual disk are lost when the virtual machine is shutdown, so each new virtual machine is started fresh.
The easiest way to create an AMI is to use an existing AMI and customise it for your own purposes. Amazon provide a number of basic AMI's for a couple of Linux distributions and Windows versions. Once you have customised your virtual machine to your liking, you take a snapshot of its virtual disk to create a new image. A good guide to the process is given at http://geekindulgence.blogspot.com/2008/10/customize-existing-amazon-machine-image.html
For rdiff-backup, an AMI has already been created that is available for you to use. This AMI includes rdiff-backup-1.3.3 and a web front end for restoring individual files.
Running the Virtual Machine
Running a virtual machine involves starting an instance from an AMI, waiting until it boots to get the DNS address of the virtual machine, attaching a volume to the virtual machine and then mounting the volume, running the backup, and then terminating the virtual machine.
To make the process simple, and to facilitate automated backups, a script, ec2-rdiff-backup.sh, is available here that manages the entire process.
To use the script, edit the parameters at the beginning to suit your particular environment. In particular, set EC2_GROUP to match the one you created earlier, EC2_VOLUME to match the volume-id of the volume you created for your backups, and RSH_KEY to point to the pem file you created as the your authorisation key.
The EC2_IMG is currently set to the rdiff-backup virtual server public image for rdiff-backup-1.3.3. This will change in future as bugs are fixed in the virtual server image and later releases of rdiff-backup are massage centres in singapore made available. To find the latest images use the command:
ec2-describe-images -a | grep rdiff-backup
This will return a list of available images with the rdiff-backup version numbers and image sub-versions. Select the appropriate AMI id (which will look like ami-xxxxxxxx) and copy it into the script.
If this is the first time you have used the EBS volume created above, you will need to initialise it. To do this, start the virtual server with the command:
ec2-rdiff-backup.sh format
This will partition and format the volume, erasing any data that was previously on it.
The script is set up to mount the EBS volume under /backup on the server. Multiple backup sets can be located on the same volume by specifying different backup names. Note that an EBS volume can only be attached to one virtual machine at a prenatal massage in singapore time, so if you are running multiple backups to a single volume you will need to coordinate them so that only one is running at a time.
Running a Backup
Running the ec2-rdiff-backup.sh script with no options will start a server, attach and mount a volume, run a backup and terminate the server using the default settings specified in the script. Most of these setting can be overridden using command line options - use 'ec2-rdiff-backup.sh -h' for more details.
Restoring Files
If you want to restore individual files, the easiest way is the use the rdiff-backup-browser web interface. To access the web interface use the command:
ec2-rdiff-backup.sh start
This will start the virtual server and attach and mount the volume using the default settings, which can be overridden by command line options. The command will return the necessary details for deals singapore accessing the virtual server. Copy and paste the http address into you web browser to access the rdiff-backup-browser web interface.
The rdiff-backup-browser web interface allows you to select the backup set and backup increment to browse. All of the files in the backup set are displayed. Click on the directories to expand their contents. Click on individual files to download the version at the time of the backup. Hover over a file or directory to view details about the owner, size and last modification date. Note - for large backup sets, displaying the initial file list will take a little time, just be patient.
Once you have finished the restore, use the:
ec2-rdiff-backup.sh stop
command to stop the virtual machine. If you have more than one virtual machine running, you will need to specify the instance id returned by the start command.
To restore entire directories, the easiest way is to run the following command on your local machine:
ec2-rdiff-backup.sh -f <file/directory to restore> \ -d <destination of restore> -r <restore as of time/increment> restore
This command will start the virtual machine, attach and mount the volume, run the restore, and then terminate the virtual machine. <restore as of time/increment> is as per rdiff-backup's time specification.
Warning
Whilst ever a virtual machine is running you are being charged by Amazon. It is your responsibility to ensure that virtual machines are terminated when no longer required. The ec2-rdiff-backup.sh script tries to ensure that virtual machines are not left running unnecessarily but there are some conditions, like loss of network connection, where it is unable to do so. No liability is accepted by the authors for charges incurred in running this script. Web 2.0 Design
