duplicity-team team mailing list archive

[Scott <question196423@xxxxxxxxxxxxxxxxxxxxx>]

Question #196423 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/196423

Description changed to:
I'm getting duplicity setup and doing a partial test upload to my hosted
webspace. I've tried several times now and am getting the same result:
it works great up until partway through the 41st *.difftar.gpg file, at
which point it fails with this error:

#######
INFO 1
 . Using archive dir: /home/firecat53/.cache/duplicity/1498ef4c9557164d69523e5aec4c78ce
 
 INFO 1
 . Using backup name: 1498ef4c9557164d69523e5aec4c78ce
 
 INFO 1
 . Import of duplicity.backends.sshbackend Succeeded
 
 INFO 1
 . Import of duplicity.backends.rsyncbackend Succeeded
 
 INFO 1
 . Import of duplicity.backends.botobackend Succeeded
 
 INFO 1
 . Import of duplicity.backends.webdavbackend Succeeded
 
 INFO 1
 . Import of duplicity.backends.hsibackend Succeeded
 
 INFO 1
 . Import of duplicity.backends.localbackend Succeeded
 
 INFO 1
 . Import of duplicity.backends.ftpsbackend Succeeded
 
 WARNING 1
 . Import of duplicity.backends.giobackend Failed: No module named gio
 
 INFO 1
 . Import of duplicity.backends.ftpbackend Succeeded
 
 INFO 1
 . Import of duplicity.backends.cloudfilesbackend Succeeded
 
  0 INFO 1
INFO 1
 . Import of duplicity.backends.imapbackend Succeeded
 
 INFO 1
 . Import of duplicity.backends.gdocsbackend Succeeded
 
 INFO 1
 . Main action: inc
 
 INFO 1
 . ================================================================================
 
 INFO 1
 . duplicity 0.6.18 (February 29, 2012)
 
 INFO 1
 . Args: /usr/bin/duplicity --use-agent -vdebug --encrypt-key 42A79D21 --sign-key E6C991E3 --full-if-older-than 60D --exclude /mnt/backup/bfd-training-data --exclude /mnt/backup/fsarchiver --exclude /mnt/backup/media --exclude /mnt/backup/usapparel --num-retries 25 --log-file /home/firecat53/dup.log /mnt/backup scp://user@xxxxxxxxxxxxxxx//backups/homeserver
 
 INFO 1
 . Linux homeserver 3.0.29-1-lts #1 SMP PREEMPT Mon Apr 23 09:41:11 CEST 2012 x86_64 AMD Sempron(tm) Processor 3400+
 
 INFO 1
 . /usr/bin/python2 2.7.3 (default, Apr 24 2012, 00:00:54)
 . [GCC 4.7.0 20120414 (prerelease)]
 
 INFO 1
 . ================================================================================
 
 INFO 1
 . Using temporary directory /tmp/duplicity-Hj3jqk-tempdir
DEBUG 1
 . Registering (mkstemp) temporary file /tmp/duplicity-Hj3jqk-tempdir/mkstemp-IHXNWE-1
 
 INFO 1
 . Temp has 490516480 available, backup will use approx 34078720.
 
 NOTICE 1
 . Local and Remote metadata are synchronized, no sync needed.
 
 DEBUG 1
 . 0 files exist on backend
 
 DEBUG 1
 . 0 files exist in cache
 
 DEBUG 1
 . Extracting backup chains from list of files: []
 
 NOTICE 1
 . Last full backup date: none
 
 NOTICE 1
 . Last full backup is too old, forcing full backup
 
 INFO 3
 backend SftpBackend
 archive-dir (() /home/firecat53/.cache/duplicity/1498ef4c9557164d69523e5aec4c78ce dir)
 orphaned-sets-num 0
 incomplete-sets-num 0
 . Collection Status
 . -----------------
 . Connecting with backend: SftpBackend
 . Archive dir: /home/firecat53/.cache/duplicity/1498ef4c9557164d69523e5aec4c78ce
 .
 . Found 0 secondary backup chains.
 . No backup chains with active signatures found
 . No orphaned or incomplete backup sets found.

<snip lots of code>

DEBUG 1
 . Removing still remembered temporary file /tmp/duplicity-Hj3jqk-tempdir/mkstemp-IHXNWE-1
 
 DEBUG 1
 . Removing still remembered temporary file /tmp/duplicity-Hj3jqk-tempdir/mktemp-bZzCOB-42
 
 INFO 1
 . Backend error detail: Traceback (most recent call last):
 .   File "/usr/bin/duplicity", line 1391, in <module>
 .     with_tempdir(main)
 .   File "/usr/bin/duplicity", line 1384, in with_tempdir
 .     fn()
 .   File "/usr/bin/duplicity", line 1354, in main
 .     full_backup(col_stats)
 .   File "/usr/bin/duplicity", line 500, in full_backup
 .     globals.backend)
 .   File "/usr/bin/duplicity", line 399, in write_multivol
 .     (tdp, dest_filename, vol_num)))
 .   File "/usr/lib/python2.7/site-packages/duplicity/asyncscheduler.py", line 145, in schedule_task
 .     return self.__run_synchronously(fn, params)
 .   File "/usr/lib/python2.7/site-packages/duplicity/asyncscheduler.py", line 171, in __run_synchronously
 .     ret = fn(*params)
 .   File "/usr/bin/duplicity", line 398, in <lambda>
 .     async_waiters.append(io_scheduler.schedule_task(lambda tdp, dest_filename, vol_num: put(tdp, dest_filename, vol_num)    ,
 .   File "/usr/bin/duplicity", line 296, in put
 .     backend.put(tdp, dest_filename)
 .   File "/usr/lib/python2.7/site-packages/duplicity/backends/sshbackend.py", line 191, in put
 .     raise BackendException("sftp put of %s (as %s) failed: %s" % (source_path.name,remote_filename,e))
 . BackendException: sftp put of /tmp/duplicity-Hj3jqk-tempdir/mktemp-bZzCOB-42 (as duplicity-full.20120507T221749Z.vol41.d    ifftar.gpg) failed:
 .
 
 ERROR 23 BackendException
 . BackendException: sftp put of /tmp/duplicity-Hj3jqk-tempdir/mktemp-bZzCOB-42 (as duplicity-full.20120507T221749Z.vol41.d    ifftar.gpg) failed:
######


Any ideas? I had trouble at first getting it to restart, but now that aspect seems to be working (although it asks for the Encryption Key passphrase, which I don't undertand because I thought with using the --sign-key parameter that it would just need the signing key passphrase, not the encryption key passphrase)

Thanks!
Scott

Edit 2: Well, after restarting it again, it died at the 2GB uploaded
point. Am I missing something obvious? I've never had any issues with
the webhosting provider before with large uploads using rsync, so I'm
fairly (not completely) certain it's not them. I'll try it to a local
hard drive and see if it does the same thing.

Edit 3: Sorry about all the edits! Well trying it on a local drive
(file:// protocol) seems to work fine. It's still going on a 30G backup
after 2.8G. So I wonder if it's my hosting provider or something in the
ssh/scp protocol? If it is the provider, why would it start working
again right away when I restart the backup? The volume size is the
default 25MB. Any ideas?

-- 
You received this question notification because you are a member of
duplicity-team, which is an answer contact for Duplicity.