← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #47972

[Bug 1556717] [NEW] live_migration_force_complete surprised error message on compute node in some situation

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

On master branch

trigger live-migration-force-complete after invoke live-migration test3

ubuntu@lm-1:~/devstack$ nova live-migration-force-complete test3 64

on the source compute node, we can see follow error trace, but it
doesn't effect running instance.

expect:

No error message come out.

actual:

2016-03-14 01:18:13.770 ERROR oslo_messaging.rpc.dispatcher [req-6d6ceb08-561b-4638-b813-0c8f8c6c4669 admin admin] Exception during message handling: Instance instance-0000002b could not be found.
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     incoming.message))
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 110, in wrapped
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     payload)
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     self.force_reraise()
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 89, in wrapped
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     return f(self, context, *args, **kw)
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 375, in decorated_function
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 5281, in live_migration_force_complete
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     self.driver.live_migration_force_complete(instance)
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 6544, in live_migration_force_complete
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     self.pause(instance)
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2519, in pause
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     self._host.get_guest(instance).pause()
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 617, in get_guest
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     self.get_domain(instance))
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 607, in get_domain
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     return self._get_domain_by_name(instance.name)
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 654, in _get_domain_by_name
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     raise exception.InstanceNotFound(instance_id=instance_name)
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher InstanceNotFound: Instance instance-0000002b could not be found.
2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher

** Affects: nova
     Importance: Undecided
     Assignee: Eli Qiao (taget-9)
         Status: New


** Tags: live-migration

** Changed in: nova
     Assignee: (unassigned) => Eli Qiao (taget-9)

** Tags added: live-migration

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1556717

Title:
  live_migration_force_complete surprised error message on compute node
  in some situation

Status in OpenStack Compute (nova):
  New

Bug description:
  On master branch

  trigger live-migration-force-complete after invoke live-migration
  test3

  ubuntu@lm-1:~/devstack$ nova live-migration-force-complete test3 64

  on the source compute node, we can see follow error trace, but it
  doesn't effect running instance.

  expect:

  No error message come out.

  actual:

  2016-03-14 01:18:13.770 ERROR oslo_messaging.rpc.dispatcher [req-6d6ceb08-561b-4638-b813-0c8f8c6c4669 admin admin] Exception during message handling: Instance instance-0000002b could not be found.
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     incoming.message))
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 110, in wrapped
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     payload)
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     self.force_reraise()
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 89, in wrapped
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     return f(self, context, *args, **kw)
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 375, in decorated_function
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 5281, in live_migration_force_complete
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     self.driver.live_migration_force_complete(instance)
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 6544, in live_migration_force_complete
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     self.pause(instance)
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2519, in pause
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     self._host.get_guest(instance).pause()
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 617, in get_guest
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     self.get_domain(instance))
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 607, in get_domain
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     return self._get_domain_by_name(instance.name)
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 654, in _get_domain_by_name
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher     raise exception.InstanceNotFound(instance_id=instance_name)
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher InstanceNotFound: Instance instance-0000002b could not be found.
  2016-03-14 01:18:13.770 TRACE oslo_messaging.rpc.dispatcher

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1556717/+subscriptions
  Thread Previous • Date Previous • • Thread Next