yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1227836] Re: Grizzly live-migration fails in Ubuntu 12.04

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Joe Gordon <1227836@xxxxxxxxxxxxxxxxxx>
Date: Mon, 13 Oct 2014 20:54:30 -0000
Reply-to: Bug 1227836 <1227836@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Grizzly isn't supported upstream anymore, closing bug.

** Changed in: nova
       Status: Incomplete => Won't Fix

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1227836

Title:
  Grizzly live-migration fails in Ubuntu 12.04

Status in OpenStack Compute (Nova):
  Won't Fix

Bug description:
  Hi guys i'm trying to use live-migration with this scenario:

  OS: Ubuntu 12.04.3
  OpenStack version: Grizzly
  Nova version: 2013.1.2-0ubuntu1~cloud0
  Shared storage with GFS2

  My migration configuration in nova-compute.conf is:

  ...
  live_migration_bandwidth=0
  live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE
  live_migration_retry_count=30
  live_migration_uri=qemu+tcp://%s/system
  ...

  When i try to migrate an instance i get some kind of RPC time out (i
  sent a couple of mails to the operators list with the subject
  "Migrating instances in grizzly"):

  nova-compute.log
  ...
  2013-09-02 15:35:45.784 4601 DEBUG nova.openstack.common.rpc.common [-] Timed out waiting for RPC response: timed out _error_callback /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py:628
  2013-09-02 15:35:45.790 4601 ERROR nova.utils [-] in fixed duration looping call
  2013-09-02 15:35:45.790 4601 TRACE nova.utils Traceback (most recent call last):
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 594, in _inner
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     self.f(*self.args, **self.kw)
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3129, in wait_for_live_migration
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     migrate_data)
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3208, in _post_live_migration
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     migration)
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 664, in network_migrate_instance_start
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     migration)
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 415, in network_migrate_instance_start
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     return self.call(context, msg, version='1.41')
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/proxy.py", line 80, in call
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     return rpc.call(context, self._get_topic(topic), msg, timeout)
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/__init__.py", line 140, in call
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     return _get_impl().call(CONF, context, topic, msg, timeout)
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py", line 798, in call
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     rpc_amqp.get_connection_pool(conf, Connection))
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 612, in call
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     rv = list(rv)
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 554, in __iter__
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     self.done()
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     self.gen.next()
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 551, in __iter__
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     self._iterator.next()
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py", line 648, in iterconsume
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     yield self.ensure(_error_callback, _consume)
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py", line 566, in ensure
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     error_callback(e)
  2013-09-02 15:35:45.790 4601 TRACE nova.utils   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py", line 629, in _error_callback
  2013-09-02 15:35:45.790 4601 TRACE nova.utils     raise rpc_common.Timeout()
  2013-09-02 15:35:45.790 4601 TRACE nova.utils Timeout: Timeout while waiting on RPC response.
  2013-09-02 15:35:45.790 4601 TRACE nova.utils
  ...

  -The VM state never changes back to ACTIVE from MIGRATING
  -The VM actually moves from one node to the other (i can see it with "virsh list" on the new node) and it still answers ping

  Any clues?

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1227836/+subscriptions