apport-hackers team mailing list archive

[Brian Murray <brian@xxxxxxxxxx>]

Brian Murray has proposed merging lp:~brian-murray/apport/check-contents-server-age into lp:apport.

Requested reviews:
  Apport upstream developers (apport-hackers)

For more details, see:
https://code.launchpad.net/~brian-murray/apport/check-contents-server-age/+merge/244071

As I was looking at working on bug 1370230 it occurred to me that we don't need to download Contents-$arch.gz just because the file on the local system is a day old, especially as those don't change very often for stable releases. Subsequently, I've added a check for the 'last-modified' header on the server to check when Contents-$arch.gz file changed. We then only download the file if it is newer than the one on disk. This may make things slightly faster and certainly uses bandwidth more efficiently.
-- 
Your team Apport upstream developers is requested to review the proposed merge of lp:~brian-murray/apport/check-contents-server-age into lp:apport.

=== modified file 'backends/packaging-apt-dpkg.py'
--- backends/packaging-apt-dpkg.py	2014-10-10 13:18:14 +0000
+++ backends/packaging-apt-dpkg.py	2014-12-08 23:28:49 +0000
@@ -905,26 +905,41 @@
                 age = None
 
             if age is None or age >= 86400:
-                url = '%s/dists/%s%s/Contents-%s.gz' % (self._get_mirror(), release, pocket, arch)
-
-                try:
-                    src = urlopen(url)
-                except IOError:
-                    # we ignore non-existing pockets, but we do crash if the
-                    # release pocket doesn't exist
-                    if pocket == '':
-                        raise
+                if age:
+                    import httplib
+                    from datetime import datetime
+                    conn = httplib.HTTPConnection("%s" % self._get_mirror().split('/')[2])
+                    conn.request("HEAD", "/ubuntu/dists/%s%s/Contents-%s.gz" %
+                        (release, pocket, arch))
+                    res = conn.getresponse()
+                    modified_str = res.getheader('last-modified')
+                    modified = datetime.strptime(modified_str, '%a, %d %b %Y %H:%M:%S %Z')
+                    if modified > datetime.fromtimestamp(st.st_mtime):
+                        update = True
                     else:
-                        continue
+                        update = False
+                else:
+                    update = True
+                if update:
+                    url = '%s/dists/%s%s/Contents-%s.gz' % (self._get_mirror(), release, pocket, arch)
+                    try:
+                        src = urlopen(url)
+                    except IOError:
+                        # we ignore non-existing pockets, but we do crash if the
+                        # release pocket doesn't exist
+                        if pocket == '':
+                            raise
+                        else:
+                            continue
 
-                with open(map, 'wb') as f:
-                    while True:
-                        data = src.read(1000000)
-                        if not data:
-                            break
-                        f.write(data)
-                src.close()
-                assert os.path.exists(map)
+                    with open(map, 'wb') as f:
+                        while True:
+                            data = src.read(1000000)
+                            if not data:
+                                break
+                            f.write(data)
+                    src.close()
+                    assert os.path.exists(map)
 
             if file.startswith('/'):
                 file = file[1:]