← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1162047] Re: lockutils.synchronized does not work properly if lock_path=none

 

** Changed in: oslo
       Status: Fix Committed => Fix Released

** Changed in: oslo
    Milestone: None => havana-3

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1162047

Title:
  lockutils.synchronized does not work properly if lock_path=none

Status in OpenStack Compute (Nova):
  Fix Released
Status in Oslo - a Library of Common OpenStack Code:
  Fix Released

Bug description:
  We discovered today that two processes will not lock properly if
  external=True and lock_path=None.

  nova/openstack/common/lockutils.py tries to use tempfile.mkdtemp() to
  create a temporary directory for tracking the locking but this doesn't
  work as tempfile.mkdtemp() returns two different temporary
  directories.  Here is an example of a simple test case running using
  locking that demonstrates the problem:

  openstack@issue147007:~/devel$ ./run_tests.sh -N nova.tests.test_sync
  Running ` python setup.py testr --slowest --testr-args='--subunit  nova.tests.test_sync'`
  running testr                                                                            
  running=OS_STDOUT_CAPTURE=1 OS_STDERR_CAPTURE=1 OS_TEST_TIMEOUT=60 ${PYTHON:-python} -m subunit.run discover -t ./ ./nova/tests --list 
  running=OS_STDOUT_CAPTURE=1 OS_STDERR_CAPTURE=1 OS_TEST_TIMEOUT=60 ${PYTHON:-python} -m subunit.run discover -t ./ ./nova/tests  --load-list /tmp/tmpsB1Gxo
  running=OS_STDOUT_CAPTURE=1 OS_STDERR_CAPTURE=1 OS_TEST_TIMEOUT=60 ${PYTHON:-python} -m subunit.run discover -t ./ ./nova/tests  --load-list /tmp/tmpo09Jxy
  ('Running modified test_sync!!!! CONF.lock_path is %s', None)                                                                                              
  ('Running modified test_sync!!!! CONF.lock_path is %s', None)                                                                                              
  ('JSBRYANT: local_lock_path is %s', '/tmp/tmp80M3nJ')                                                                                                      
  Thread 1 Start                                                                                                                                             
  Thread 1 End                                                                                                                                               
  nova.tests.test_sync.TestLocking.test_one (subunit.RemotedTestCase)                                                                                        
  nova.tests.test_sync.TestLocking.test_one ... ok                                                                                                           
  ('JSBRYANT: local_lock_path is %s', '/tmp/tmp898Hj_')                                                                                                      
  Thread 2 Start                                                                                                                                             
  Thread 2 End                                                                                                                                               
  nova.tests.test_sync.TestLocking.test_two (subunit.RemotedTestCase)                                                                                        
  nova.tests.test_sync.TestLocking.test_two ... ok                                                                                                           
  Slowest Tests                                                                                                                                              
  Test id                                    Runtime (s)                                                                                                     
  -----------------------------------------  -----------                                                                                                     
  nova.tests.test_sync.TestLocking.test_two  120.102                                                                                                         
  nova.tests.test_sync.TestLocking.test_one   60.062                                                                                                         

  ----------------------------------------------------------------------
  Ran 2 tests in 122.583s 

  Note the two test cases shouldn't run in parallel but they do as the
  locking isn't working.  The output shows the two different lock
  directories which were created here:

                      if external and not CONF.disable_process_locking:
                          LOG.debug(_('Attempting to grab file lock "%(lock)s" '
                                      'for method "%(method)s"...'),
                                    {'lock': name, 'method': f.__name__})
                          cleanup_dir = False

                          # We need a copy of lock_path because it is non-local
                          local_lock_path = lock_path
                          if not local_lock_path:
                              local_lock_path = CONF.lock_path

                          if not local_lock_path:
                              cleanup_dir = True
                              local_lock_path = tempfile.mkdtemp()

                          print("JSBRYANT: local_lock_path is %s",
  local_lock_path)

                          if not os.path.exists(local_lock_path):
                              fileutils.ensure_tree(local_lock_path)

  
  This can be worked around by either specifying the lock_path on the call to synchronized or by updating the conf file to have a lock_path.  Regardless, this seems like something that cause unpredictable behavior for users who don't get those options into their conf files.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1162047/+subscriptions