duplicity-team team mailing list archive

[Michael Terry <michael.terry@xxxxxxxxxxxxx>]

Michael Terry has proposed merging lp:~mterry/duplicity/webdav-fixes into lp:duplicity.

Requested reviews:
  duplicity-team (duplicity-team)

For more details, see:
https://code.launchpad.net/~mterry/duplicity/webdav-fixes/+merge/223183

This branch fixes two issues I saw when testing the webdav backend:

1) Errors like the following: "Attempt 1 failed. BackendException: File /tmp/duplicity-LQ1a0i-tempdir/mktemp-u2aiyX-2 not found locally after get from backend"

These were caused by the _get() method not calling setdata() on the local path object, so the rest of the code thought it didn't exist.

2) Some odd issues from stale responses/data.  We have a couple places in webdavbackend.py where we close the connection before making a request because of this problem.  But I've changed it to do it every time, more reliably, by putting a _close() call inside the request() method.

With this, the webdav backend seems fine to me.
-- 
https://code.launchpad.net/~mterry/duplicity/webdav-fixes/+merge/223183
Your team duplicity-team is requested to review the proposed merge of lp:~mterry/duplicity/webdav-fixes into lp:duplicity.

=== modified file 'duplicity/backends/webdavbackend.py'
--- duplicity/backends/webdavbackend.py	2014-05-07 21:41:31 +0000
+++ duplicity/backends/webdavbackend.py	2014-06-15 20:25:37 +0000
@@ -161,7 +161,7 @@
             and self.conn.host == self.parsed_url.hostname: return
 
         log.Info("WebDAV create connection on '%s'" % (self.parsed_url.hostname))
-        if self.conn: self.conn.close()
+        self._close()
         # http schemes needed for redirect urls from servers
         if self.parsed_url.scheme in ['webdav','http']:
             self.conn = httplib.HTTPConnection(self.parsed_url.hostname, self.parsed_url.port)
@@ -174,13 +174,15 @@
             raise FatalBackendException("WebDAV Unknown URI scheme: %s" % (self.parsed_url.scheme))
 
     def _close(self):
-        self.conn.close()
+        if self.conn:
+            self.conn.close()
 
     def request(self, method, path, data=None, redirected=0):
         """
         Wraps the connection.request method to retry once if authentication is
         required
         """
+        self._close() # or we get previous request's data or exception
         self.connect()
 
         quoted_path = urllib.quote(path,"/:~")
@@ -309,7 +311,6 @@
         for i in range(1,len(dirs)):
             d="/".join(dirs[0:i+1])+"/"
 
-            self._close() # or we get previous request's data or exception
             self.headers['Depth'] = "1"
             response = self.request("PROPFIND", d)
             del self.headers['Depth']
@@ -318,12 +319,10 @@
 
             if response.status == 404:
                 log.Info("Creating missing directory %s" % d)
-                self._close() # or we get previous request's data or exception
 
                 res = self.request("MKCOL", d)
                 if res.status != 201:
                     raise BackendException("WebDAV MKCOL %s failed: %s %s" % (d,res.status,res.reason))
-                self._close()
 
     def taste_href(self, href):
         """
@@ -366,15 +365,12 @@
         url = self.directory + remote_filename
         response = None
         try:
-            target_file = local_path.open("wb")
             response = self.request("GET", url)
             if response.status == 200:
                 #data=response.read()
-                target_file.write(response.read())
+                local_path.writefileobj(response)
                 #import hashlib
                 #log.Info("WebDAV GOT %s bytes with md5=%s" % (len(data),hashlib.md5(data).hexdigest()) )
-                assert not target_file.close()
-                response.close()
             else:
                 status = response.status
                 reason = response.reason

=== modified file 'testing/manual/backendtest'
--- testing/manual/backendtest	2014-05-11 11:50:12 +0000
+++ testing/manual/backendtest	2014-06-15 20:25:37 +0000
@@ -52,6 +52,7 @@
     def setUp(self):
         super(ManualBackendBase, self).setUp()
         self.set_global('num_retries', 1)
+        self.set_global('ssl_no_check_certificate', True)
         self.setBackendInfo()
         if self.password is not None:
             self.set_environ("FTP_PASSWORD", self.password)