duplicity-team team mailing list archive

[Michael Terry <michael.terry@xxxxxxxxxxxxx>]

Michael Terry has proposed merging lp:~mterry/duplicity/cloudfiles-10k into lp:duplicity.

Requested reviews:
  duplicity-team (duplicity-team)
Related bugs:
  Bug #832149 in Duplicity: "Uploads to Rackspace fail silently"
  https://bugs.launchpad.net/duplicity/+bug/832149

For more details, see:
https://code.launchpad.net/~mterry/duplicity/cloudfiles-10k/+merge/73185
-- 
https://code.launchpad.net/~mterry/duplicity/cloudfiles-10k/+merge/73185
Your team duplicity-team is requested to review the proposed merge of lp:~mterry/duplicity/cloudfiles-10k into lp:duplicity.

=== modified file 'duplicity/backends/cloudfilesbackend.py'
--- duplicity/backends/cloudfilesbackend.py	2011-02-12 15:11:34 +0000
+++ duplicity/backends/cloudfilesbackend.py	2011-08-28 19:10:27 +0000
@@ -119,7 +119,13 @@
         for n in range(1, globals.num_retries+1):
             log.Info("Listing '%s'" % (self.container))
             try:
-                keys = self.container.list_objects()
+                # Cloud Files will return a max of 10,000 objects.  We have
+                # to make multiple requests to get them all.
+                objs = self.container.list_objects()
+                keys = objs
+                while len(objs) == 10000:
+                    objs = self.container.list_objects(marker=keys[-1])
+                    keys += objs
                 return keys
             except self.resp_exc, resperr:
                 log.Warn("Listing of '%s' failed (attempt %s): CloudFiles returned: %s %s"