yahoo-eng-team team mailing list archive

[OpenStack Infra <1716344@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/513243
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1606467b29969eb45efbb56c1b148a4a6f53c5cf
Submitter: Zuul
Branch:    master

commit 1606467b29969eb45efbb56c1b148a4a6f53c5cf
Author: jichenjc <jichenjc@xxxxxxxxxx>
Date:   Wed Oct 18 11:20:51 2017 +0800

    Downgrade log for keystone verify client fail
    
    Under some circumstances the keystone verify process might fail
    but we are able to proceed because it's client setting error,
    so we don't need to report an exception log in the log file to
    confuse admin, instead, use an info log.
    
    In the reported bug, the issue is that nova is configured for
    the 'internal' identity endpoint but the nova code does not
    pass an interface, so KSA defaults to 'public' which fails.
    This is fixed with I2204c8bed8936d5bed0f410284d2a563f84e7100
    but not something we can backport, so this is a simple change
    to make the logging less annoying.
    
    Closes-Bug: 1716344
    
    Change-Id: I67c9f648f85de364de443e2a0535ddd361c14661


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1716344

Title:
  Nova-API uses Keystone's public endpoint for project id verification

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) pike series:
  Confirmed

Bug description:
  I have setup a fresh HA deployment of OpenStack Pike on Ubuntu 16.04.
  I recognized in the logs that Nova fails during vm creation with the
  following exception:

  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity [req-6efab9e1-78f5-4e85-8247-686ff4f3568c dddfba8e02f746799a6408a523e6cd25 ed2d2efd86dd40e7a45491d8502318d3 - default default] Unable to contact keystone to verify project_id: SSLError: SSL exception connecting to https://os-cloud.mycompany.com:5000/v3/projects/ed2d2efd86dd40e7a45491d8502318d3: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",)
  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity Traceback (most recent call last):
  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/identity.py", line 42, in verify_project_id
  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity     raise_exc=False)
  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity   File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 845, in get
  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity     return self.request(url, 'GET', **kwargs)
  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity   File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line 101, in inner
  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity     return wrapped(*args, **kwargs)
  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity   File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 703, in request
  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity     resp = send(**kwargs)
  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity   File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 765, in _send_request
  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity     raise exceptions.SSLError(msg)
  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity SSLError: SSL exception connecting to https://os-cloud.mycompany.com:5000/v3/projects/ed2d2efd86dd40e7a45491d8502318d3: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",)
  2017-09-11 09:31:28.909 5604 ERROR nova.api.openstack.identity

  Keystone's public endpoint should only visible to external clients.
  All internal OpenStack services should use the internalURL for
  authentication purposes. I think my configuration is correct. The
  "auth_url" point to Keystone's internal URL, whereas "auth_uri" points
  to Keystone's public endpoint. I want to avoid https based
  communication for my internal cloud services.

  $ openstack endpoint list | grep keystone
  | 00a22bfee72141ddadacd0e357161075 | RegionOne | keystone     | identity        | True    | internal  | http://os-identity.mycompany.com:5000/v3                        |
  | 7178e534cb4e4c5e9a67066ff3e9c433 | RegionOne | keystone     | identity        | True    | public    | https://os-cloud.mycompany.com:5000/v3                          |
  | f5ed3bba70274d7fa03f2ceaab96c3c9 | RegionOne | keystone     | identity        | True    | admin     | http://os-identity.mycompany.com:35357/v3                       |

  ################
  nova.conf
  ################
  ...
  [keystone_authtoken]
  auth_type = password
  auth_uri = http://os-cloud.mycompany.com:5000
  auth_url = http://os-identity:35357
  memcached_servers = os-memcache:11211
  password = novapass
  project_domain_name = default
  project_name = service
  user_domain_name = default
  username = nova
  ...

  Can someone please have a look?

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1716344/+subscriptions