yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1515830] [NEW] Class Openssl.Ssl.Syscallerror

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Matthew Taylor <matthew.taylor@xxxxxxxxxxxxxxxxxxxxx>
Date: Fri, 13 Nov 2015 01:29:04 -0000
Reply-to: Bug 1515830 <1515830@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

During heat stack creation, we had encountered an unexpected API error
from nova:

Resource Create Failed: Resources.Kolla Nodes: Clientexception:
Resources[0].Resources.Kolla Node: Unexpected Api Error. Please Report
This At Http: //Bugs.Launchpad.Net/Nova/ And Attach The Nova Api Log If
Possible. <Class 'cd /ro'> (Http 500) (Request-Id: Req-
24a122c9-0ac1-4921-9e52-0878a58f32b7)

This happens every time we attempt to launch the heat stack for kolla.

You should be able to replicate this by following kolla's cluster
creation with heat guide here:

http://docs.openstack.org/developer/kolla/devenv-heat.html#create-the-
stack

Issue occurred shortly after running:

heat stack-create -f kollacluster.yaml -e local.yaml kolla-cluster

Note: kollacluster.yaml remains un-edited, however local.yaml contains
basic information such as our external network ID's etc.

We are using Liberty. Here is the current nova/heat RPM's installed:

[root@controller01]$ rpm -qa|egrep '(nova|heat)'
openstack-heat-common-5.0.0-1.el7.noarch
openstack-nova-cells-12.0.0-1.el7.noarch
openstack-heat-api-5.0.0-1.el7.noarch
openstack-nova-common-12.0.0-1.el7.noarch
openstack-nova-scheduler-12.0.0-1.el7.noarch
python-novaclient-2.30.1-1.el7.noarch
openstack-nova-objectstore-12.0.0-1.el7.noarch
openstack-heat-api-cfn-5.0.0-1.el7.noarch
openstack-nova-cert-12.0.0-1.el7.noarch
openstack-nova-conductor-12.0.0-1.el7.noarch
openstack-nova-api-12.0.0-1.el7.noarch
openstack-heat-engine-5.0.0-1.el7.noarch
python-nova-12.0.0-1.el7.noarch
openstack-nova-novncproxy-12.0.0-1.el7.noarch
openstack-nova-network-12.0.0-1.el7.noarch
openstack-heat-api-cloudwatch-5.0.0-1.el7.noarch
python-heatclient-0.8.0-1.el7.noarch
openstack-nova-console-12.0.0-1.el7.noarch

>From nova-api.log:

2015-11-13 00:57:03.298 25129 DEBUG keystoneclient.session [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] RESP: [200] date: Fri, 13 Nov 2015 00:57:03 GMT connection: keep-alive content-type: application/json; charset=UTF-8 content-length: 762 x-openstack-request-id: req-e0251fb3-a788-482e-8719-cefb89bb0947
2015-11-13 00:57:03.307 25129 DEBUG oslo_concurrency.lockutils [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] Acquired semaphore "neutron_admin_auth_token_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:198
2015-11-13 00:57:03.308 25129 DEBUG keystoneclient.auth.identity.v3.base [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] Making authentication request to https://control-internal.flashcloud.com.au:35357/v3/auth/tokens get_auth_ref /usr/lib/python2.7/site-packages/keystoneclient/auth/identity/v3/base.py:188
2015-11-13 00:57:03.309 25129 DEBUG oslo_concurrency.lockutils [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] Releasing semaphore "neutron_admin_auth_token_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:211
2015-11-13 00:57:03.310 25129 ERROR nova.api.openstack.extensions [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] Unexpected exception in API method
2015-11-13 00:57:03.316 25129 INFO nova.api.openstack.wsgi [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
2015-11-13 00:57:03.317 25129 DEBUG nova.api.openstack.wsgi [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
2015-11-13 00:57:03.318 25129 INFO nova.osapi_compute.wsgi.server [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] 172.19.18.2 "POST /v2/1621e5f8695d4af88cb5e4230d4dc244/servers HTTP/1.1" status: 500 len: 429 time: 1.0944760

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1515830

Title:
  Class Openssl.Ssl.Syscallerror

Status in OpenStack Compute (nova):
  New

Bug description:
  During heat stack creation, we had encountered an unexpected API error
  from nova:

  Resource Create Failed: Resources.Kolla Nodes: Clientexception:
  Resources[0].Resources.Kolla Node: Unexpected Api Error. Please Report
  This At Http: //Bugs.Launchpad.Net/Nova/ And Attach The Nova Api Log
  If Possible. <Class 'cd /ro'> (Http 500) (Request-Id: Req-
  24a122c9-0ac1-4921-9e52-0878a58f32b7)

  This happens every time we attempt to launch the heat stack for kolla.

  You should be able to replicate this by following kolla's cluster
  creation with heat guide here:

  http://docs.openstack.org/developer/kolla/devenv-heat.html#create-the-
  stack

  Issue occurred shortly after running:

  heat stack-create -f kollacluster.yaml -e local.yaml kolla-cluster

  Note: kollacluster.yaml remains un-edited, however local.yaml contains
  basic information such as our external network ID's etc.

  We are using Liberty. Here is the current nova/heat RPM's installed:

  [root@controller01]$ rpm -qa|egrep '(nova|heat)'
  openstack-heat-common-5.0.0-1.el7.noarch
  openstack-nova-cells-12.0.0-1.el7.noarch
  openstack-heat-api-5.0.0-1.el7.noarch
  openstack-nova-common-12.0.0-1.el7.noarch
  openstack-nova-scheduler-12.0.0-1.el7.noarch
  python-novaclient-2.30.1-1.el7.noarch
  openstack-nova-objectstore-12.0.0-1.el7.noarch
  openstack-heat-api-cfn-5.0.0-1.el7.noarch
  openstack-nova-cert-12.0.0-1.el7.noarch
  openstack-nova-conductor-12.0.0-1.el7.noarch
  openstack-nova-api-12.0.0-1.el7.noarch
  openstack-heat-engine-5.0.0-1.el7.noarch
  python-nova-12.0.0-1.el7.noarch
  openstack-nova-novncproxy-12.0.0-1.el7.noarch
  openstack-nova-network-12.0.0-1.el7.noarch
  openstack-heat-api-cloudwatch-5.0.0-1.el7.noarch
  python-heatclient-0.8.0-1.el7.noarch
  openstack-nova-console-12.0.0-1.el7.noarch

  From nova-api.log:

  2015-11-13 00:57:03.298 25129 DEBUG keystoneclient.session [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] RESP: [200] date: Fri, 13 Nov 2015 00:57:03 GMT connection: keep-alive content-type: application/json; charset=UTF-8 content-length: 762 x-openstack-request-id: req-e0251fb3-a788-482e-8719-cefb89bb0947
  2015-11-13 00:57:03.307 25129 DEBUG oslo_concurrency.lockutils [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] Acquired semaphore "neutron_admin_auth_token_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:198
  2015-11-13 00:57:03.308 25129 DEBUG keystoneclient.auth.identity.v3.base [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] Making authentication request to https://control-internal.flashcloud.com.au:35357/v3/auth/tokens get_auth_ref /usr/lib/python2.7/site-packages/keystoneclient/auth/identity/v3/base.py:188
  2015-11-13 00:57:03.309 25129 DEBUG oslo_concurrency.lockutils [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] Releasing semaphore "neutron_admin_auth_token_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:211
  2015-11-13 00:57:03.310 25129 ERROR nova.api.openstack.extensions [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] Unexpected exception in API method
  2015-11-13 00:57:03.316 25129 INFO nova.api.openstack.wsgi [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  2015-11-13 00:57:03.317 25129 DEBUG nova.api.openstack.wsgi [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  2015-11-13 00:57:03.318 25129 INFO nova.osapi_compute.wsgi.server [req-24a122c9-0ac1-4921-9e52-0878a58f32b7 0b5f4f6455c440de87641b99bcf1fccc 1621e5f8695d4af88cb5e4230d4dc244 - - -] 172.19.18.2 "POST /v2/1621e5f8695d4af88cb5e4230d4dc244/servers HTTP/1.1" status: 500 len: 429 time: 1.0944760

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1515830/+subscriptions
Follow ups

[Bug 1515830] Re: Class Openssl.Ssl.Syscallerror
From: Launchpad Bug Tracker, 2016-01-30