launchpad-reviewers team mailing list archive

[Māris Fogels <mfogels@xxxxxxxxx>]

Māris Fogels has proposed merging lp:~mars/launchpad/fix-ec2-shutdown-617598 into lp:launchpad/devel.

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)
Related bugs:
  #617598 ec2 instance does not shut down after test failures
  https://bugs.launchpad.net/bugs/617598


Hi,

This branch fixes a problem with competing shutdown calls in the ec2 testrunner stepping on each other's toes, causing the instance to keep running for a full 8 hours.  See bug 617598 for details.

On salgado's advice this backs out the failsafe shutdown code just in case someone else writes a shutdown call somewhere in the future and encounters this same problem.  I backed out r11224 of devel in it's entirety.

While making the fix I dropped the failsafe timeout from 8 hours to 5.  The suite runs inside four hours, so this should not be a problem.

I will run 'ec2 land' in order to test this change.


Maris
-- 
https://code.launchpad.net/~mars/launchpad/fix-ec2-shutdown-617598/+merge/32633
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~mars/launchpad/fix-ec2-shutdown-617598 into lp:launchpad/devel.

=== modified file 'lib/devscripts/ec2test/builtins.py'
--- lib/devscripts/ec2test/builtins.py	2010-07-29 19:52:31 +0000
+++ lib/devscripts/ec2test/builtins.py	2010-08-13 21:53:44 +0000
@@ -307,7 +307,7 @@
             open_browser=open_browser, pqm_email=pqm_email,
             include_download_cache_changes=include_download_cache_changes,
             instance=instance, launchpad_login=instance._launchpad_login,
-            timeout=480)
+            timeout=300)
 
         instance.set_up_and_run(postmortem, attached, runner.run_tests)
 

=== modified file 'lib/devscripts/ec2test/testrunner.py'
--- lib/devscripts/ec2test/testrunner.py	2010-07-25 10:43:37 +0000
+++ lib/devscripts/ec2test/testrunner.py	2010-08-13 21:53:44 +0000
@@ -318,7 +318,11 @@
     def configure_system(self):
         user_connection = self._instance.connect()
         if self.timeout is not None:
-            user_connection.perform("sudo -b shutdown -h +%s" % self.timeout)
+            # Activate a fail-safe shutdown just in case something goes
+            # really wrong with the server or suite.
+            user_connection.perform(
+                "echo sudo shutdown -h now | at today + %d minutes"
+                % self.timeout)
         as_user = user_connection.perform
         # Set up bazaar.conf with smtp information if necessary
         if self.email or self.message: