← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1598860] Re: live migration failing when nova-conductor is started before other services

 

Reviewed:  https://review.openstack.org/339072
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=dc6b3ab6ef59b4d3f9e540f1cc07326494cd8565
Submitter: Jenkins
Branch:    master

commit dc6b3ab6ef59b4d3f9e540f1cc07326494cd8565
Author: John Garbutt <john.garbutt@xxxxxxxxxxxxx>
Date:   Thu Jul 7 16:20:27 2016 +0100

    Don't cache RPC pin when service_version is 0
    
    Its possible for RPC calls to be made before a nova-compute process
    starts up. When this happens we cache the RPC pin of kilo. This causes
    breakages in live-migrate, but in Mitaka we only work with Liberty.
    
    Change-Id: I5f1e0904c34c4fe26fc75998441602925d2f0593
    Closes-Bug: #1598860


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1598860

Title:
  live migration failing when nova-conductor is started before other
  services

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  Deploying OpenStack current master via kolla source, with kvm and
  ceph/rbd enabled and attempting to perform a live-migration, it
  becomes stuck in state migrating with this appearing in the nova-
  compute log.

  Looking at nova git log, i'm seeing a bunch of live-migration changes
  in the last few days.  I suspect there might have been a regression.

  2016-07-04 10:00:15.643 1 INFO nova.compute.resource_tracker [req-1f6df1d8-a88a-4f27-8ce2-609ea25da4e2 - - - - -] Compute_service record updated for compute01:compute01
  2016-07-04 10:00:56.731 1 ERROR root [req-a2b97b2c-f479-4b7c-b3d9-142c1ab1b25f b3bedc85b7674ce2b7e0589a7427dc81 22ab55c3ffe343639d872b1e4db6abb4 - - -] Original exception being dropped: ['Traceback (most recent call last):\n', '  File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/exception_wrapper.py", line 66, in wrapped\n    return f(self, context, *args, **kw)\n', '  File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/compute/utils.py", line 608, in decorated_function\n    *args, **kwargs)\n', '  File "/usr/lib64/python2.7/inspect.py", line 980, in getcallargs\n    \'arguments\' if num_required > 1 else \'argument\', num_total))\n', 'TypeError: live_migration() takes exactly 7 arguments (6 given)\n']
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server [req-a2b97b2c-f479-4b7c-b3d9-142c1ab1b25f b3bedc85b7674ce2b7e0589a7427dc81 22ab55c3ffe343639d872b1e4db6abb4 - - -] Exception during message handling
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/exception_wrapper.py", line 71, in wrapped
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server     f, self, context, *args, **kw)
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/exception_wrapper.py", line 85, in _get_call_dict
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server     context, *args, **kw)
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server   File "/usr/lib64/python2.7/inspect.py", line 980, in getcallargs
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server     'arguments' if num_required > 1 else 'argument', num_total))
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server TypeError: live_migration() takes exactly 7 arguments (6 given)
  2016-07-04 10:00:56.732 1 ERROR oslo_messaging.rpc.server

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1598860/+subscriptions


References