yahoo-eng-team team mailing list archive

[Ihar Hrachyshka <1698355@xxxxxxxxxxxxxxxxxx>]

Neutron fix: https://review.openstack.org/#/c/474575/

Also oslo.serialization that resulted in memory exhaustion:
https://review.openstack.org/#/c/475052/

** Also affects: oslo.serialization
   Importance: Undecided
       Status: New

** Changed in: oslo.serialization
   Importance: Undecided => Critical

** Changed in: oslo.serialization
       Status: New => Confirmed

** Changed in: heat
       Status: New => Confirmed

** Changed in: oslo.serialization
     Assignee: (unassigned) => Ihar Hrachyshka (ihar-hrachyshka)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1698355

Title:
  py35 dsvm job failing with RemoteDisconnected error

Status in heat:
  Confirmed
Status in neutron:
  In Progress
Status in oslo.serialization:
  Confirmed

Bug description:
  traceback:

  2017-06-16 10:24:47.339195 | 2017-06-16 10:24:47.338 | 
  2017-06-16 10:24:47.340517 | 2017-06-16 10:24:47.340 | heat_integrationtests.scenario.test_autoscaling_lbv2.AutoscalingLoadBalancerv2Test.test_autoscaling_loadbalancer_neutron
  2017-06-16 10:24:47.342125 | 2017-06-16 10:24:47.341 | ------------------------------------------------------------------------------------------------------------------------
  2017-06-16 10:24:47.343471 | 2017-06-16 10:24:47.343 | 
  2017-06-16 10:24:47.344919 | 2017-06-16 10:24:47.344 | Captured traceback:
  2017-06-16 10:24:47.346272 | 2017-06-16 10:24:47.346 | ~~~~~~~~~~~~~~~~~~~
  2017-06-16 10:24:47.347614 | 2017-06-16 10:24:47.347 |     b'Traceback (most recent call last):'
  2017-06-16 10:24:47.348873 | 2017-06-16 10:24:47.348 |     b'  File "/opt/stack/new/heat/heat_integrationtests/common/test.py", line 376, in _stack_delete'
  2017-06-16 10:24:47.350049 | 2017-06-16 10:24:47.349 |     b'    success_on_not_found=True)'
  2017-06-16 10:24:47.351627 | 2017-06-16 10:24:47.351 |     b'  File "/opt/stack/new/heat/heat_integrationtests/common/test.py", line 357, in _wait_for_stack_status'
  2017-06-16 10:24:47.352791 | 2017-06-16 10:24:47.352 |     b'    fail_regexp):'
  2017-06-16 10:24:47.353977 | 2017-06-16 10:24:47.353 |     b'  File "/opt/stack/new/heat/heat_integrationtests/common/test.py", line 321, in _verify_status'
  2017-06-16 10:24:47.355411 | 2017-06-16 10:24:47.355 |     b'    stack_status_reason=stack.stack_status_reason)'
  2017-06-16 10:24:47.356920 | 2017-06-16 10:24:47.356 |     b"heat_integrationtests.common.exceptions.StackBuildErrorException: Stack AutoscalingLoadBalancerv2Test-1133164547 is in DELETE_FAILED status due to 'Resource DELETE failed: ConnectFailure: resources.sec_group: Unable to establish connection to http://10.1.43.45:9696/v2.0/security-group-rules/8d33f0cf-d473-455a-8fe2-978c64af5e0d: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))'"
  2017-06-16 10:24:47.358227 | 2017-06-16 10:24:47.357 |     b''

  http://logs.openstack.org/65/473765/1/check/gate-heat-dsvm-functional-
  convg-mysql-lbaasv2-py35-ubuntu-
  xenial/e07f32f/console.html#_2017-06-16_10_24_47_356920

  
  heat engine log:

  http://logs.openstack.org/65/473765/1/check/gate-heat-dsvm-functional-
  convg-mysql-lbaasv2-py35-ubuntu-
  xenial/e07f32f/logs/screen-h-eng.txt.gz?level=INFO#_Jun_16_10_24_22_312023

  
  In the same job nova is failing to connect to neutron with the same error

  
  http://logs.openstack.org/65/473765/1/check/gate-heat-dsvm-functional-convg-mysql-lbaasv2-py35-ubuntu-xenial/e07f32f/logs/screen-n-api.txt.gz?level=ERROR#_Jun_16_10_24_22_633655

  
  It seems to be happening for security-groups and ports/floating-ip stuff mostly.

  
  Not sure if this is a neutronclient/urllib3 issue(I see a new openstacksdk release[1])or something specific to changes merged recently to neuton[2].

  
  [1] https://github.com/openstack/requirements/commit/1b30d517efd442867888359e4619d822f13a3cf2

  [2] https://review.openstack.org/#/q/topic:bp/push-notifications

To manage notifications about this bug go to:
https://bugs.launchpad.net/heat/+bug/1698355/+subscriptions