duplicity-team team mailing list archive

[edso <question212302@xxxxxxxxxxxxxxxxxxxxx>]

Question #212302 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/212302

    Status: Open => Answered

edso proposed the following answer:
On 25.10.2012 18:05, Keith Cornwell wrote:
> New question #212302 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/212302
> 
> I running a full backup to a local USB disk. After 364G and ~18 hours it became stuck in a loop. All output to stdout and stderr stopped. The last thing on stdout was this:
> 
> AsyncScheduler: scheduling task for asynchronous execution
> Processed volume 3923
> Writing /mnt/external/backup-Shared/duplicity-full.20121024T210212Z.vol3923.difftar.gpg
> Deleting /tmp/duplicity-0AQWHS-tempdir/mktemp-IwcN_x-3924
> AsyncScheduler: task execution done (success: True)
> 
> The difftar file isn't up to the volsize of 100M.
> 
> The last thing written to stderr was:
> Exception AttributeError: AttributeError("'_DummyThread' object has no attribute '_Thread__block'",) in <module 'threading' from '/usr/lib/python2.7/threading.pyc'> ignored
> This is repeated many times.
> 
> I attached to the duplicity process with strace and I'm seeing the following repeating continuously:
> 
> select(0, NULL, NULL, NULL, {0, 1000})  = 0 (Timeout)
> gettimeofday({1351180030, 542252}, NULL) = 0
> select(0, NULL, NULL, NULL, {0, 2000})  = 0 (Timeout)
> gettimeofday({1351180030, 544707}, NULL) = 0
> select(0, NULL, NULL, NULL, {0, 4000})  = 0 (Timeout)
> gettimeofday({1351180030, 549159}, NULL) = 0
> select(0, NULL, NULL, NULL, {0, 8000})  = 0 (Timeout)
> gettimeofday({1351180030, 557631}, NULL) = 0
> select(0, NULL, NULL, NULL, {0, 16000}) = 0 (Timeout)
> gettimeofday({1351180030, 574106}, NULL) = 0
> select(0, NULL, NULL, NULL, {0, 32000}) = 0 (Timeout)
> gettimeofday({1351180030, 606550}, NULL) = 0
> select(0, NULL, NULL, NULL, {0, 33995}) = 0 (Timeout)
> gettimeofday({1351180030, 640995}, NULL) = 0
> gettimeofday({1351180030, 641246}, NULL) = 0
> gettimeofday({1351180030, 641443}, NULL) = 0
> 
> command line:
> duplicity --verbosity info --name opus-shared --volsize 100 --asynchronous-upload /filestore01/Shared file:///mnt/external/backup-Shared
> 
> Linux opus 3.2.0-32-generic-pae #51-Ubuntu SMP Wed Sep 26 21:54:23 UTC 2012 i686 athlon i386 GNU/Linux
> ubuntu server 12.04 LTS
> duplicity 0.6.18
> python 2.7.3
> gpg (GnuPG) 1.4.11
> 
> If I stop this is there any way to resume the backup from this point?
> 
> I would be grateful for any help.
> 

my guess is your ram fills up until no space is left anymore. if that's the case, that is a known issue.
you are probably using duplicity packaged from ubuntu repository.. don't do that! why? here is an excerpt from the README of the next release

"
A NOTE ON GnuPGInterface.py AND MULTIPLE GPG PROCESSES:

GnuPGInterface is used to access GPG from duplicity.  The original works
quite well and has no bugs, however, we have patched the one used in
duplicity.  Why?  Duplicity is not perfect, yet, and has a problem when
handling long chains of incremental backup or restore operations.  The
problem is that the waitpid() call only happens after all the iterations
complete, and with a long chain, that can be a long while.  Unless the
waitpid() call is made, the child process remains active.  Duplicity's
GnuPGInterface is patched to start an immediate threaded waitpid() for
each GPG task, thus harvesting the task and freeing it's resources in a
timely manner.  This does not affect the operation of duplicity, merely
frees resources on time.

Why the note? Some package maintainers remove duplicity's GnuPGInterface
in error, obviously unknowing of this issue and patch duplicity to use
the old unmaintained unpatched GnuPGInterface interface again. So, if
you have the problem that lots of GPG tasks are hanging around, check
and see if this has been done in your distro, and if so, report this
matter as a bug to the distro or package maintainer.

As of october 2012 we pull the handbrake and refactor our code and rename the class to gpginterface in the hope that package maintainers will stumble over it and stop this problematic behaviour for good.
"

so essentially you should either install duplicity from .tgz avail on
http://duplicity.nongnu.org

wrt resuming your backup. duplicity is able to but i wouldn't advise you
to do that (there are still hickups sometimes). if you decide to resume
you should do a verify immediately after to make sure your backup is
flawless.

..ede/duply.net

-- 
You received this question notification because you are a member of
duplicity-team, which is an answer contact for Duplicity.