duplicity-team team mailing list archive

[Rodrigo Alvarez <question248111@xxxxxxxxxxxxxxxxxxxxx>]

Question #248111 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/248111

    Status: Needs information => Open

Rodrigo Alvarez gave more information on the question:
I upgraded to duplicity 0.7.06 and reran and observed the exact same
behavior.

1) I run a fresh full backup of a TimeMachine Sparse bundle set that uses about 700GB. This takes about 14 hrs to complete. 
2) Without any changes to the primary set I rerun a second backup, intending to get a zero-sized incremental backup. Instead, duplicity starts backing up about 300 GB. 

I've confirmed that the files in the primary set are not changing (by
looking at their modification date, which for some files that are being
re-backed up is in 2014).

This problem had previously manifested in a smaller data set and a work
around (to avoid using necessary space in the backend) was to build the
primary set progressively.  Previously, the backup duplication started
after about 92K files. This time it starts at about 49K files.

More specifically, the set consists of 180288, about half of which are
invisible files (.AppleDouble) which are being excluded.  The bulk is a
folder named 'bands' which contains 90132 files, all off which are at
most 8.0MBytes (except about 54 of them that are smaller).  From this,
it seems the problem on the second backup pass starts after duplicity
has gone through 49794 bands, or 398,352 Mbytes.   This number seems
consistent with the point at which the previous backup set would also
start duplicating (but I'm estimating from memory here).

I'd be very happy to run experiments or provide more information
(including logs) to help debug this issue.

Is it my specific system or am I stressing duplicity way beyond what
normal people use it for?

A bug about this issue had previously been created (as per the Related
Bug above).

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.