FAQs

From RdiffBackupWiki

Jump to: navigation, search

See the main site FAQ as well.

Contents

[edit]

Why does my (first?) backup run so slow, after upgrading from v0.10.1 to v0.13.3?

A: Firstly, file metadata must be read by traversing the file system. On later runs it will be read from the metadata file (which only versions 0.11.1 or later build). Secondly, the development versions are often slower. Before 0.13.x becomes 0.14.0 it will be profiled and optimized.

[edit]

Status of Rdiff-backup on windows

The native rdiff-backup win32 binary works out of the box but to use it with remote linux hosts some additional steps must be completeted found in BackupFromWindowsToLinux

For other alternatives you can get some packages here:

http://solutionsfirst.com.au/~dave/backup/ however I recommend you read this blog entry: 

http://katastrophos.net/andre/blog/?p=19

for details on some of the problems and a patch to 'fix' win32 -> win32 backups. However, this patch is no longer necessary for rdiff-backup versions 1.1.8 and above. Also, version 1.1.12 has some important fixes to make rdiff-backup "just work" on Cygwin.

[edit]

When I backup, I get a lot of ListErrors. Why? Is my backup OK?

  ListError {local_file} [[Errno 1]] Operation not permitted: {file_on_remote_mount}

There is some mismatch of file permissions between the user running the rdiff-backup and the user that owns the file being backed up.

My backup did not complete successfully when I got this error. I was running rdiff-backup on one machine, that had mounted another machines NFS share. The user name/user_id map in /etc/passwd did not match on the two machines. Running the script as root worked fine.

[edit]

Files that change while being backed up cause errors. How serious are they? Do I just not get the recent changes?

Yes, changing files will probably cause Update Errors, which means you won't get the changes for that day. See the official error policy at: http://rdiff-backup.stanford.edu/error_policy.html

[edit]

How do I run rdiff-backup with python 2.3 (or 2.2)? The rpm I have requires python 2.2 (or 2.3)

Use a source RPM. When you compile the source rpm, the resulting binary RPM will require whatever version of python you used to compile it. So if you have python 2.3 installed and make a rdiff-backup binary RPM, the RPM will require python 2.3. So just use 

 rpmbuild --rebuild rdiff-backup-x.x.x-x.src.rpm

=== I get "UpdateError filename File changed from regular file before signature" every time I back up on files that have long since ceased to exist. This also happens on a file I tried to manually remove from the backup. ===

See FileChanged in the ErrorsAndSolutions section.

[edit]

I have a backup server with lots of space that does NOT have rdiff-backup installed. I also have lots of local space. Will permissions etc. be preserved if I do something like .. 

  rdiff-backup --exclude /BACKUP / /BACKUP
  rsync --archive --delete /BACKUP user@backup-server:/home/user/

Yes, rdiff-backup saves metadata in extra files so this should work. Although it would probably be easier to use something like SHFS (http://shfs.sourceforge.net) to access the remote storage, if it is not available via NFS anyway.

[edit]

Not making remote connections?

When I try to run 

  # rdiff-backup /var/www/html michael@bla.domain.tld:1202:~/backup

I get an error "Unable to create directory michael@bla.domain.tld:1202:~/backup"

Here is some other output from testing: 

  # rdiff-backup --test-server michael@bla.domain.tld:1202:~/backup
  No remote connections specified

  #rdiff-backup --test-server michael@12.123.123.123:1202:/
  No remote connections specified

I installed rdiff-backup on Fedora Core 5 using 

  # yum -y install rdiff-backup

I can backup to a directory on the same machine just fine, but when I try to specifiy a remote machine, it still thinks I mean a local directory named michael@... I have searched and searched for a solution, but cannot find anyone else having this problem. Any suggestions?

Thanks, Michael.

Dear Michael, command syntax is: 

rdiff-backup [[options]] [[[[user@]]host1.foo]::source_directory]

Notice the two consecutive ::

You cannot specify TCP ports the way you have done.

[edit]

How do I remove files from the backup set / I didn't like answer #8 in the official FAQ

This method is very dangerous and shouldn't be used, unless the files that you want to remove are causing your backup drive to run out of space and your only alternative to removing those files is removing entire increments.

IMPORTANT: Properly speaking, you should do step 4 for every increment of mirror_metadata. Rdiff-backup prior to 1.1.1 does not mind having extra mirror_metadata entries for files that are removed from the backup set this way, except in the most recent version of mirror_metadata. However, at 1.1.1 the mirror_metadata handling changed -- rdiff-backup now diffs the metadata files -- and it's unknown whether having extra entries in these diff'd files will affect restore operations. (Technical note: the mirror_metadata diffs are NOT using the same method as file diffs. They are not rdiff delta files, but plain text files (and no, they are not ordinary text diffs either). Because of this, it is safe to hand-edit them, so if you need to you can do step 4 on these diffs.)

1. Check the time -- make sure it is not close to time for a scheduled run of rdiff-backup. Also make sure rdiff-backup is not running.

2. Go into your mirror target directory and delete the file or directory there.

3. Go into rdiff-backup-data/increments on the target and delete all traces of the file/directory there. Important! If you are removing a directory, make sure you find and remove all of the *.dir files for it as well! If it's a file, make sure you find and remove all of the *.missing files (if there are any). Be careful not to remove anything that isn't related to the thing you're trying to remove, or you might lose the ability to restore other files.

4. Important step! (and WARNING this is untested with rdiff-backup 1.1.1 or later) Go back up into rdiff-backup-data and gunzip the latest mirror-metadata file. Edit the mirror_metadata file in a well-behaved text editor (WARNING! Do not use pico or nano or anything else that might automatically do line wrapping!) and remove all references to the file or directory you deleted. Be very careful not to mess up the format of the file.

[edit]

How do restores work?

  • What happens when I have to restore a file?

If the file hasn't changed between current and the version of the file you want, then you simply need to copy the current files from the rdiff-backup repository. Nothing more has to be done.
If it has changed, see the next point...


  • Does the system have to restore all the reverse diffs for a file?

If the file has changed (i.e. You want a version that is not the same as the current version, and the version you want exists in the rdiff-backup repository) it will take the current version of the file and use the meta-data stored to tell it how to apply all the reverse differences files that apply back to the date you requested, provided it exists. For RDiff-Backup to be successful in restoring the file, it has to have all three parts:

  1. The current version of the file as it existed when RDiff-Backup was last run.
  2. The meta data that tells the system if/when/how to apply the reverse diffs.
  3. All the reverse diffs themselves.

Reverse diffs have to be applied in the reverse order they were made. (i.e. The newest RDiff is applied, then the next oldest and so on, back to the very oldest reverse diff that applies to the version of the file requested.)


  • What if there are dozens or even hundreds?

Yes, the system has to apply all the reverse diffs that apply to the "version" of the file you requested. If there were 200 reverse diffs, because the file had changed over 200 rdiff-backup sessions, it will have to apply all 200 reverse diffs to get to the version of the file you want. If any of the three parts of the system; current file, meta-data, or reverse diffs are missing the process will break and you won't get your file.


  • What if only one is broken, is the whole process of "restoring" the file broken?

There are ways to attempt to manually salvage the file, but these are far outside the scope of this document. Suffice it to say, that if any of the parts (file/meta-data/rdiffs) needed are missing, RDiff-Backup isn't going to be able to restore it automatically and all bets are off. You'll be in deep weeds and if you're lucky you might be able to get parts of your data back. Perhaps if you're really super lucky and the missing reverse diffs overlap others *and* you can finagle the restore process, you might get everything back. Or, if it's just not your day, you won't get jack, you'll get fired, your dog will bite you and you'll get rabies...


  • Isn't it dangerous to have to rely on all those reverse diffs, especially when they're being applied serially, and every single one of the reverse diffs has to apply properly and in order to get back to the version I want?

Yes, it is "dangerous" - though every definition of dangerous depends on your perspective. (Just ask a BASE jumper about what's considered dangerous.) The design decision was to only keep the differences with no intermediate snapshots of files. (Also, due to limitations in the rsync libraries, it's impossible to merge rdiffs which might allow us to reduce the number of independent reverse diffs we have to apply.)

While we're certainly not trying to convince you to use RDiff-Backup and agree with our reasoning on what's best and/or reasonable, we think reasonable trade-offs were made on managing the resources used vs the advantages of redundancy.

[edit]

OK, I like most of what I hear, but how can I be sure the whole system retains its integrity?

  • Is there a way to test all the parts of the system and make sure they all work, and work properly?
  • For example, can I have the system "self test" the archive and let me know if any parts of it fail?


Certainly. The "--verify-at-time xyz" switch is your friend. This switch, in essence, does a full restore and check of the file to the time specified in "xyz."

In brief, it takes the current version of the file, and then uses the meta-data and applicable reverse diffs to roll the file back to the date specified. (i.e. xyz) It then re-calculates the SHA-1 hash for the re-created file. It then checks that newly calculated SHA-1 hash with the SHA-1 hash it stored for this file when it was backed up back on the date that corresponds with "xyz."

  • If any part of the process fails or the SHA-1 hashes don't match, rdiff-backup will exit with a non-zero result. (And it should generate errors to the console...)
  • If meta-data is damaged, and it can't figure out how to apply the rdiffs, you should get an error message.
  • If after rolling the file backward to date xyz, the check-sums don't match, you'll get an error.

Thus, to test the integrity of every piece of the system, pick a date for "xyz" that is at least as old as the oldest rdiff session. This should, by requirement, apply every reverse diff in the repository and all the meta-data.

While a successful results of a "--verify-at-time xyz" isn't sufficient to ensure that someone hasn't tampered with the rdiff-repository in an attempt, for example, to modify executable files - it is very strong evidence that chance or bad-luck hasn't damaged the system. Random collisions for the same file in the SHA-1 checksum are vanishingly small. (i.e. Two very similar files having the same SHA-1 checksum but not being equal, by simple chance (not malicious design), is exceedingly unlikely.)

Retrieved from "http://wiki.rdiff-backup.org/wiki/index.php/FAQs"
Personal tools