yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

** Also affects: nova/pike
   Importance: Undecided
       Status: New

** Also affects: nova/queens
   Importance: Undecided
       Status: New

** Changed in: nova
       Status: New => Triaged

** Changed in: nova/queens
       Status: New => Triaged

** Changed in: nova/pike
       Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1752152

Title:
  Attach Volume Fails with secure call to cinder

Status in OpenStack Compute (nova):
  Triaged
Status in OpenStack Compute (nova) pike series:
  Triaged
Status in OpenStack Compute (nova) queens series:
  Triaged
Status in python-cinderclient:
  New

Bug description:
  It is found that when cinder endpoint is configured to use https,
  attach volume flow fails with the stack trace seen below (seen in nova
  api log) because it fails to make a secure call from nova to cinder.
  Secure calls perform certificate validation and in this particular
  flow, certificate validation is completely skipped

  File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3971, in attach_volume
  2018-02-27 08:16:51.338 1324 ERROR      cinder.is_microversion_supported(context, '3.44')
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 138, in is_microversion_supported
  2018-02-27 08:16:51.338 1324 ERROR      _check_microversion(url, microversion)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 86, in _check_microversion
  2018-02-27 08:16:51.338 1324 ERROR      max_api_version = cinder_client.get_highest_client_server_version(url)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 126, in get_highest_client_server_version
  2018-02-27 08:16:51.338 1324 ERROR      min_server, max_server = get_server_version(url)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 109, in get_server_version
  2018-02-27 08:16:51.338 1324 ERROR      response = requests.get(version_url)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/api.py", line 72, in get
  2018-02-27 08:16:51.338 1324 ERROR      return request('get', url, params=params, **kwargs)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/api.py", line 58, in request
  2018-02-27 08:16:51.338 1324 ERROR      return session.request(method=method, url=url, **kwargs)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 502, in request
  2018-02-27 08:16:51.338 1324 ERROR      resp = self.send(prep, **send_kwargs)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 612, in send
  2018-02-27 08:16:51.338 1324 ERROR      r = adapter.send(request, **kwargs)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 504, in send
  2018-02-27 08:16:51.338 1324 ERROR      raise ConnectionError(e, request=request)
  2018-02-27 08:16:51.338 1324 ERROR  ConnectionError: HTTPSConnectionPool(host='ip9-114-192-132.pok.stglabs.ibm.com', port=9000): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)",),))

  This is a regression bug introduced as part of changeset
  https://review.openstack.org/#/c/469579/, which was merged way back in
  June 2017. As part of this changeset, a new function namely
  _check_microversion was introduced, which then makes a cinderclient
  call , which finally makes a cinder https REST api call without
  passing the certificate. This leads to the problem listed above.

  https://github.com/openstack/nova/blob/stable/queens/nova/volume/cinder.py#L75
  https://github.com/openstack/nova/blob/stable/queens/nova/volume/cinder.py#L86

  https://github.com/openstack/python-cinderclient/blob/stable/queens/cinderclient/client.py#L126
  https://github.com/openstack/python-cinderclient/blob/stable/queens/cinderclient/client.py#L109

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1752152/+subscriptions