launchpad-reviewers team mailing list archive

[Stuart Bishop <stuart.bishop@xxxxxxxxxxxxx>]

Stuart Bishop has proposed merging lp:~stub/launchpad/trivial into lp:launchpad.

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)
Related bugs:
  Bug #951401 in Launchpad itself: "parse-ppa-apache-logs failing (missing files)"
  https://bugs.launchpad.net/launchpad/+bug/951401
  Bug #1263002 in Launchpad itself: "Twisted feature flag support fails typecasting when updating"
  https://bugs.launchpad.net/launchpad/+bug/1263002

For more details, see:
https://code.launchpad.net/~stub/launchpad/trivial/+merge/237003

It is normal for Librarian files to exist on disk but not in Swift, because we upload the files from disk to Swift asynchronously.

Only warn about files that don't exist in Swift if they are older than 1 week, which means the librarian-feed-swift process is failing.
-- 
https://code.launchpad.net/~stub/launchpad/trivial/+merge/237003
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~stub/launchpad/trivial into lp:launchpad.

=== modified file 'lib/lp/services/librarianserver/librariangc.py'
--- lib/lp/services/librarianserver/librariangc.py	2014-08-26 05:06:38 +0000
+++ lib/lp/services/librarianserver/librariangc.py	2014-10-03 07:15:48 +0000
@@ -811,9 +811,19 @@
     if next_wanted_content_id == content_id:
         next_wanted_content_id = get_next_wanted_content_id()
     while next_wanted_content_id is not None:
-        log.error(
-            "LibraryFileContent {0} exists in the database but was not "
-            "found in Swift.".format(next_wanted_content_id))
+        # The entry exists in the database but not in Swift. This is
+        # normal, as there is lag between uploading files to disk and
+        # migrating them into Swift. The important case, where files
+        # are missing and exist neither on disk nor in Swift, is reported
+        # earlier. Still, we should catch if the librarian-feed-swift
+        # has not run recently. Report an error if the file is older
+        # than one week and doesn't exist in Swift.
+        path = get_file_path(next_wanted_content_id)
+        if os.path.exists(path) and (os.stat(path).st_ctime
+                                     < time.time() - (7 * 24 * 60 * 60)):
+            log.error(
+                "LibraryFileContent {0} exists in the database and disk "
+                "but was not found in Swift.".format(next_wanted_content_id))
         next_wanted_content_id = get_next_wanted_content_id()
 
     log.info(