← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #71436

[Bug 1752152] [NEW] Attach Volume Fails with secure call to cinder

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

It is found that when cinder endpoint is configured to use https, attach
volume flow fails with the stack trace seen below (seen in nova api log)
because it fails to make a secure call from nova to cinder. Secure calls
perform certificate validation and in this particular flow, certificate
validation is completely skipped

File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3971, in attach_volume
2018-02-27 08:16:51.338 1324 ERROR      cinder.is_microversion_supported(context, '3.44')
2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 138, in is_microversion_supported
2018-02-27 08:16:51.338 1324 ERROR      _check_microversion(url, microversion)
2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 86, in _check_microversion
2018-02-27 08:16:51.338 1324 ERROR      max_api_version = cinder_client.get_highest_client_server_version(url)
2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 126, in get_highest_client_server_version
2018-02-27 08:16:51.338 1324 ERROR      min_server, max_server = get_server_version(url)
2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 109, in get_server_version
2018-02-27 08:16:51.338 1324 ERROR      response = requests.get(version_url)
2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/api.py", line 72, in get
2018-02-27 08:16:51.338 1324 ERROR      return request('get', url, params=params, **kwargs)
2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/api.py", line 58, in request
2018-02-27 08:16:51.338 1324 ERROR      return session.request(method=method, url=url, **kwargs)
2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 502, in request
2018-02-27 08:16:51.338 1324 ERROR      resp = self.send(prep, **send_kwargs)
2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 612, in send
2018-02-27 08:16:51.338 1324 ERROR      r = adapter.send(request, **kwargs)
2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 504, in send
2018-02-27 08:16:51.338 1324 ERROR      raise ConnectionError(e, request=request)
2018-02-27 08:16:51.338 1324 ERROR  ConnectionError: HTTPSConnectionPool(host='ip9-114-192-132.pok.stglabs.ibm.com', port=9000): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)",),))

This is a regression bug introduced as part of changeset
https://review.openstack.org/#/c/469579/, which was merged way back in
June 2017. As part of this changeset, a new function namely
_check_microversion was introduced, which then makes a cinderclient call
, which finally makes a cinder https REST api call without passing the
certificate. This leads to the problem listed above.

https://github.com/openstack/nova/blob/stable/queens/nova/volume/cinder.py#L75
https://github.com/openstack/nova/blob/stable/queens/nova/volume/cinder.py#L86

https://github.com/openstack/python-cinderclient/blob/stable/queens/cinderclient/client.py#L126
https://github.com/openstack/python-cinderclient/blob/stable/queens/cinderclient/client.py#L109

** Affects: nova
     Importance: Undecided
         Status: New

** Affects: python-cinderclient
     Importance: Undecided
         Status: New

** Project changed: cinder => nova

** Also affects: python-cinderclient
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1752152

Title:
  Attach Volume Fails with secure call to cinder

Status in OpenStack Compute (nova):
  New
Status in python-cinderclient:
  New

Bug description:
  It is found that when cinder endpoint is configured to use https,
  attach volume flow fails with the stack trace seen below (seen in nova
  api log) because it fails to make a secure call from nova to cinder.
  Secure calls perform certificate validation and in this particular
  flow, certificate validation is completely skipped

  File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3971, in attach_volume
  2018-02-27 08:16:51.338 1324 ERROR      cinder.is_microversion_supported(context, '3.44')
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 138, in is_microversion_supported
  2018-02-27 08:16:51.338 1324 ERROR      _check_microversion(url, microversion)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 86, in _check_microversion
  2018-02-27 08:16:51.338 1324 ERROR      max_api_version = cinder_client.get_highest_client_server_version(url)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 126, in get_highest_client_server_version
  2018-02-27 08:16:51.338 1324 ERROR      min_server, max_server = get_server_version(url)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 109, in get_server_version
  2018-02-27 08:16:51.338 1324 ERROR      response = requests.get(version_url)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/api.py", line 72, in get
  2018-02-27 08:16:51.338 1324 ERROR      return request('get', url, params=params, **kwargs)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/api.py", line 58, in request
  2018-02-27 08:16:51.338 1324 ERROR      return session.request(method=method, url=url, **kwargs)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 502, in request
  2018-02-27 08:16:51.338 1324 ERROR      resp = self.send(prep, **send_kwargs)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 612, in send
  2018-02-27 08:16:51.338 1324 ERROR      r = adapter.send(request, **kwargs)
  2018-02-27 08:16:51.338 1324 ERROR    File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 504, in send
  2018-02-27 08:16:51.338 1324 ERROR      raise ConnectionError(e, request=request)
  2018-02-27 08:16:51.338 1324 ERROR  ConnectionError: HTTPSConnectionPool(host='ip9-114-192-132.pok.stglabs.ibm.com', port=9000): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)",),))

  This is a regression bug introduced as part of changeset
  https://review.openstack.org/#/c/469579/, which was merged way back in
  June 2017. As part of this changeset, a new function namely
  _check_microversion was introduced, which then makes a cinderclient
  call , which finally makes a cinder https REST api call without
  passing the certificate. This leads to the problem listed above.

  https://github.com/openstack/nova/blob/stable/queens/nova/volume/cinder.py#L75
  https://github.com/openstack/nova/blob/stable/queens/nova/volume/cinder.py#L86

  https://github.com/openstack/python-cinderclient/blob/stable/queens/cinderclient/client.py#L126
  https://github.com/openstack/python-cinderclient/blob/stable/queens/cinderclient/client.py#L109

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1752152/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next