yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1656075] Re: DiscoveryFailure when trying to get resource providers from the scheduler report client

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Matt Riedemann <mriedem@xxxxxxxxxx>
Date: Sat, 14 Jan 2017 16:06:03 -0000
Reply-to: Bug 1656075 <1656075@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Also affects: nova/newton
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1656075

Title:
  DiscoveryFailure when trying to get resource providers from the
  scheduler report client

Status in OpenStack Compute (nova):
  Confirmed
Status in OpenStack Compute (nova) newton series:
  New

Bug description:
  I noticed this in a TripleO job:

  http://logs.openstack.org/04/419604/1/check/gate-puppet-openstack-
  integration-4-scenario004-tempest-centos-7/5d95a8c/logs/nova/nova-
  compute.txt.gz#_2017-01-12_18_53_43_459

  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager [req-59098025-7c99-41b2-aaa9-0e5714770b3a - - - - -] Error updating resources for node centos-7-osic-cloud1-s3500-6631948.
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager Traceback (most recent call last):
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6537, in update_available_resource_for_node
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     rt.update_available_resource(context, nodename)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 540, in update_available_resource
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     self._update_available_resource(context, resources)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     return f(*args, **kwargs)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 564, in _update_available_resource
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     self._init_compute_node(context, resources)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 451, in _init_compute_node
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     self.scheduler_client.update_resource_stats(self.compute_node)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 60, in update_resource_stats
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     self.reportclient.update_resource_stats(compute_node)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 37, in __run_method
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     return getattr(self.instance, __name)(*args, **kwargs)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 476, in update_resource_stats
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     compute_node.hypervisor_hostname)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 296, in _ensure_resource_provider
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     rp = self._get_resource_provider(uuid)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 53, in wrapper
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     return f(self, *a, **k)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 209, in _get_resource_provider
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     resp = self.get("/resource_providers/%s" % uuid)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 174, in get
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     endpoint_filter=self.ks_filter, raise_exc=False)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 710, in get
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     return self.request(url, 'GET', **kwargs)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/positional/__init__.py", line 101, in inner
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     return wrapped(*args, **kwargs)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 467, in request
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     auth_headers = self.get_auth_headers(auth)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 770, in get_auth_headers
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     return auth.get_headers(self, **kwargs)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/keystoneauth1/plugin.py", line 90, in get_headers
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     token = self.get_token(session)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 90, in get_token
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     return self.get_access(session).auth_token
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 136, in get_access
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     self.auth_ref = self.get_auth_ref(session)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 196, in get_auth_ref
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     self._plugin = self._do_create_plugin(session)
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 191, in _do_create_plugin
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager     raise exceptions.DiscoveryFailure('Could not determine a suitable URL '
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager DiscoveryFailure: Could not determine a suitable URL for the plugin
  2017-01-12 18:53:43.459 15495 ERROR nova.compute.manager 

  Looks like another type of keystoneauth1 exception that we're not
  handling in the safe_connect() decorator. At some point we'll probably
  just want to handle all KSA failures globally...

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1656075/+subscriptions
References

[Bug 1656075] [NEW] DiscoveryFailure when trying to get resource providers from the scheduler report client
From: Matt Riedemann, 2017-01-12