yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1346424] Re: Baremetal node id not supplied to driver

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Dan Prince <dprince@xxxxxxxxxx>
Date: Thu, 24 Jul 2014 16:58:22 -0000
Reply-to: Bug 1346424 <1346424@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Project changed: tripleo => nova

** Also affects: tripleo
   Importance: Undecided
       Status: New

** Changed in: tripleo
     Assignee: (unassigned) => Dan Prince (dan-prince)

** Changed in: tripleo
   Importance: Undecided => Critical

** Changed in: tripleo
       Status: New => In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1346424

Title:
  Baremetal node id not supplied to driver

Status in OpenStack Compute (Nova):
  Triaged
Status in tripleo - openstack on openstack:
  In Progress

Bug description:
  A random overcloud baremetal node fails to boot during check-tripleo-
  overcloud-f20. Occurs intermittently.

  Full logs:

  http://logs.openstack.org/26/105326/4/check-tripleo/check-tripleo-overcloud-f20/9292247/
  http://logs.openstack.org/81/106381/2/check-tripleo/check-tripleo-overcloud-f20/ca8a59b/
  http://logs.openstack.org/08/106908/2/check-tripleo/check-tripleo-overcloud-f20/e9894ca/

  
  Seed's nova-compute log shows this exception:

  Jul 21 13:46:07 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 ERROR oslo.messaging.rpc.dispatcher [req-9f090bea-a974-4f3c-ab06-ebd2b7a5c9e6 ] Exception during message handling: Baremetal node id not supplied to driver for 'e13f2660-b72d-4a97-afac-64ff0eecc448'
  Jul 21 13:46:07 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
  Jul 21 13:46:07 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply
  Jul 21 13:46:07 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
  Jul 21 13:46:07 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch
  Jul 21 13:46:07 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
  Jul 21 13:46:07 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch
  Jul 21 13:46:07 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
  Jul 21 13:46:07 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/exception.py", line 88, in wrapped
  Jul 21 13:46:07 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     payload)
  Jul 21 13:46:07 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
  Jul 21 13:46:07 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/exception.py", line 71, in wrapped
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     return f(self, context, *args, **kw)
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/compute/manager.py", line 291, in decorated_function
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     pass
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/compute/manager.py", line 277, in decorated_function
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/compute/manager.py", line 341, in decorated_function
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/compute/manager.py", line 319, in decorated_function
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     kwargs['instance'], e, sys.exc_info())
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/compute/manager.py", line 307, in decorated_function
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/compute/manager.py", line 1950, in build_and_run_instance
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     node, limits)
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/openstack/common/lockutils.py", line 325, in inner
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     return f(*args, **kwargs)
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/compute/manager.py", line 1917, in do_build_and_run_instance
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     if self.driver.macs_for_instance(instance):
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 223, in macs_for_instance
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     node_uuid = self._require_node(instance)
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/nova/lib/python2.7/site-packages/nova/virt/baremetal/driver.py", line 188, in _require_node
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher     % instance['uuid'])
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher NovaException: Baremetal node id not supplied to driver for 'e13f2660-b72d-4a97-afac-64ff0eecc448'
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.981 3608 TRACE oslo.messaging.rpc.dispatcher
  Jul 21 13:46:08 host-192-168-1-236 nova-compute[3608]: 2014-07-21 13:46:07.999 3608 ERROR oslo.messaging._drivers.common [req-9f090bea-a974-4f3c-ab06-ebd2b7a5c9e6 ] Returning exception Baremetal node id not supplied to driver for 'e13f2660-b72d-4a97-afac-64ff0eecc448' to caller

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1346424/+subscriptions