Rdiff-backup-rollup

From RdiffBackupWiki

This script collects all of the files related to a particular rdiff-backup increment, wraps them up into a single tarball, and optionally removes them from the rdiff-backup-data dir. These single tarballs have two advantages: they are easier to move to external storage, and they incur less filesystem overhead.

update

I have now confirmed that archives created by this script can be re-integrated into rdiff-backup-data, and restores can use the data successfully. The warning below still applies, though ;)

WARNING:

This script seems to work, but it really needs proper testing. I have tested it on a real backup set without any apparent problems (I can still run restores and nothing complains), but I have not yet tried to restore from one of the archives this script creates.

My motivation for writing this script was to move backups so old we'll probably never need them; I wanted to have the files on hand just in case, but honestly, I can't see them ever being used... and if we do need them, I fully expect it'll take a few hours of wrangling to get the rdiff-backup-data tree properly assembled for a restore. So, in short, don't use this script unless your requirements are similar to mine.

#!/bin/bash

#this script pulls all of the rdiff-backup increment files for a given
#date and time into a single tar file, for easy archiving.
#
#usage: rdiff-backup-rollup <increment time> into <destinationfile.tar>
#
#TODO:
# - gunzip all of the zipped components before archiving, then use gzip
#   argument to tar. (probably better compression this way)
# - support date ranges or "rollup-older-than"
# - more robust checking of starting environment, to lessen chance of fuckup
#   (especially make sure the destination isn't a subdir of the current dir;
#   as that would suck).

increment=$1
destdir=$3

#check some conditions first...
if [[ $destdir == "" || $increment == "" || $2 != "into" ]]; then
        echo ""
        echo "Usage: rdiff-backup-rollup <increment time> into <destination_dir>"
        echo "e.g. rdiff-backup-rollup 2008-10-04T03:30:05-07:00 into /var/tmp"
        echo "will create /var/tmp/2008-10-04T03:30:05-07:00.rdiff-backup-increment.tar"
        echo ""
        exit
fi

if [[ `pwd | sed 's/.*\///'` != "rdiff-backup-data" ]]; then
        echo ""
        echo "You must run this command from within an rdiff-backup-data directory."
        echo ""
        exit
fi

#good enough for now; let's go:
echo ""
echo "About to create $destdir/$increment.rdiff-backup-increment.tar from all"
echo "files in the current directory (`pwd`) and all its subdirs matching"
echo "*$increment* -- is this OK?"
echo ""
echo "Press CTRL+C now to abort! Otherwise, press enter to continue."
echo ""
echo "(You will be prompted again before the rdiff-backup increment is removed"
echo "from this directory)"
echo ""
read

echo "working..."
#create the tar (don't use gzip, since all of the files are already compressed)
tar --force-local -cf "$destdir/$increment.rdiff-backup-increment.tar" *$increment*

#append to the tar all matching files
#(start in increments/ to avoid duplicating the ones we used to create the tar)
find increments/ -name "*$increment*" -print0 | xargs -0 tar --force-local -rf "$destdir/$increment.rdiff-backup-increment.tar"

echo "done creating tar; checking..."
tarfilecount=`tar --force-local -tf "$destdir/$increment.rdiff-backup-increment.tar" | wc -l`
#FIXME: this will cause problems if filenames have newlines in them
#(but seriously, who does that?)
findfilecount=`find -name "*$increment*" -type f | wc -l`

if [[ $tarfilecount == $findfilecount ]]; then
        echo ""
        echo "The tar contains the right number of things (sorry, no actual data"
        echo "verification yet); do you want to go ahead and remove this increment"
        echo "from rdiff-backup-data? If you do, you'll have to restore it from"
        echo "the tarball if/when you want to restore files from this time period"
        echo "or earlier. (Type 'yes' to remove the files)"
        read delete
        if [[ "$delete" == "yes" ]]; then
                find . -name "*$increment*" -type f -print0 | xargs -0 rm
#FIXME: this leaves empty dirs behind because I couldn't think of a quick & dirty way of clearing them
        else
                echo "Files NOT deleted. To delete them later, do this:"
                echo "find . -name "*$increment*" -type f -print0 | xargs -0 rm"
        fi
else
#I hope this error message makes thedailywtf.com ;)
        echo "Oops."
        exit
fi

echo ""
echo "Important Note! tar doesn't like colons (:) in filenames, so"
echo "you'll have to give the filename of this archive as"
echo "./$increment.rdiff-backup-increment.tar"
echo "(dot-slash in front) when you're working with it later."
echo ""