duplicity-team team mailing list archive

[Rodrigo Alvarez <question248111@xxxxxxxxxxxxxxxxxxxxx>]

Question #248111 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/248111

    Status: Answered => Open

Rodrigo Alvarez is still having a problem:
Dear Edso,

I ran another test after updating to duplicity 0.23 and the issue
persists.

1. I upgraded duplicity, cleaned all out cache and tmp directories and ran a backup. 
2. Ran verify 
3. Ran a 2nd backup pass 
4. Ran a 3rd backup pass. 

On the 2nd backup pass, I still observed that about ~180 GB of data was
re-added (as listed in the log and confirmed by looking at the volumes),
even though the 1st pass (and now the verify) state that those files
where there.  The 3rd pass behaved as expected and did not add any
additional files or data.

You had previously asked about which file system I'm running. I have
ext4 on both source and destination.

I've uploaded a new set of logs (~6MB) )to
http://sira.duckdns.org/temp/duplicity_logs_photos_bakup_2014_05_08.tar.gz

Poking into the code, I found that the comparison that triggers the
'delta add' is happening around line 172 of
pyshared/duplicity/diffdir.py, but without much context, I'm worry to
launch into a hacking spree.  I'd be happy to add any debug statements
and rerun.

Best,

+R

On May 7, 2014, at 2:47 AM, edso <question248111@xxxxxxxxxxxxxxxxxxxxx>
wrote:

> Your question #248111 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/248111
> 
> edso proposed the following answer:
> On 07.05.2014 11:21, edso wrote:
>> Question #248111 on Duplicity changed:
>> https://answers.launchpad.net/duplicity/+question/248111
>> 
>> edso proposed the following answer:
>> On 07.05.2014 11:06, Rodrigo Alvarez wrote:
>>> Question #248111 on Duplicity changed:
>>> https://answers.launchpad.net/duplicity/+question/248111
>>> 
>>> Rodrigo Alvarez posted a new comment:
>>> Hi Edso,  Thanks for the follow up.
>>> 
>>> I looked at the commutative file size of all the duplicity-inc* volumes
>>> from my second pass and `du -hs` reports 177GB;  Not what we'd expect
>>> from rdiff/librsync rolling checksums.    I've upgraded to duplicity
>>> 0.63, cleared all volumes, caches and temp files and launched a new
>>> backup set.    I'll report back in a couple of days to see if this fixes
>>> the "Comparing to None" issue.
>>> 
>> 
>> hmm, usually the changed size is reflected in the incr bkp size. but in
>> your case it also stated lot's of new files, so your finding seems
>> consequential.
>> 
>> TIP:
>> do a verify between the runs! it'll inform you if it thinks there were changes.
>> 
> 
> ok, just had another look at my duplicity stats. i confused ChangedFileSize with RawDeltaSize 
> it's pretty self explanatory, but i add some hints anyway
> Delta's are modified existing block chunks. if there were only chunks added to a file you'll get no Delta's.
> 
> --------------[ Backup Statistics ]--------------
> StartTime 1399424418.52 (Wed May  7 03:00:18 2014)
> EndTime 1399424651.44 (Wed May  7 03:04:11 2014)
> ElapsedTime 232.92 (3 minutes 52.92 seconds)
> SourceFiles 113455
> SourceFileSize 4560812785 (4.25 GB)
> NewFiles 281
> NewFileSize 9192719 (8.77 MB)
> DeletedFiles 8
> ChangedFiles 81
> ChangedFileSize 46077585 (43.9 MB)
> ChangedDeltaSize 0 (0 bytes) 
> ^-- no changes in the changed files only additions
> DeltaEntries 370 
> ^-- count of chunks changed in total
> RawDeltaSize 9569326 (9.13 MB) 
> ^-- amount of bytes to backup in total
> TotalDestinationSizeChange 6696412 (6.39 MB) 
> ^-- not sure why it's less here, compression?
> Errors 7 
> ^-- i had some errors ;) there are some files inaccessible due to permissions in my backup which i don't care about
> -------------------------------------------------
> 
> ..ede/duply.net
> 
> -- 
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.launchpad.net/duplicity/+question/248111/+confirm?answer_id=5
> 
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/duplicity/+question/248111
> 
> You received this question notification because you asked the question.

-- 
You received this question notification because you are a member of
duplicity-team, which is an answer contact for Duplicity.