yahoo-eng-team team mailing list archive

[Brian Haley <1900763@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

This is related to Octavia issue
https://storyboard.openstack.org/#!/story/2008254

When the OVN Octavia Provider driver calls into octavia-lib to update
the status of a loadbalancer, for example, the code in
helper:_update_status_to_octavia() it might fail:

DEBUG ovn_octavia_provider.helper [-] Updating status to octavia: {'listeners': [{'id': '7033179d-2ddb-4714-9c06-b7d399498238', 'provisioning_status>
ERROR ovn_octavia_provider.helper [-] Error while updating the load balancer status: 'NoneType' object has no attribute 'update': octavia_lib.api.dr>
ERROR ovn_octavia_provider.helper [-] Unexpected exception in request_handler: octavia_lib.api.drivers.exceptions.UpdateStatusError: ('The status up>
ERROR ovn_octavia_provider.helper Traceback (most recent call last):
ERROR ovn_octavia_provider.helper   File "/opt/stack/ovn-octavia-provider/ovn_octavia_provider/helper.py", line 318, in _update_status_to_octavia
ERROR ovn_octavia_provider.helper     self._octavia_driver_lib.update_loadbalancer_status(status)
ERROR ovn_octavia_provider.helper   File "/usr/local/lib/python3.8/dist-packages/octavia_lib/api/drivers/driver_lib.py", line 121, in update_loadbal>
ERROR ovn_octavia_provider.helper     raise driver_exceptions.UpdateStatusError(
ERROR ovn_octavia_provider.helper octavia_lib.api.drivers.exceptions.UpdateStatusError: 'NoneType' object has no attribute 'update'

This is failing because the listener associated with the loadbalancer
was not found, it's DB transaction was in-flight.  That's the related
Octavia issue from above, but a fix for that will not solve the problem.

A side-effect is this listener is now "stuck":

$ openstack loadbalancer listener delete 7033179d-2ddb-4714-9c06-b7d399498238
Load Balancer 2cc1d429-b176-48e5-adaa-946be2af0d51 is immutable and cannot be updated. (HTTP 409) (Request-ID: req-0e1e53ac-4db9-4779-b1f3-11210fe46f7f)

The provider driver needs to retry the operation, typically even the
very next call succeeds.

** Affects: neutron
     Importance: High
     Assignee: Brian Haley (brian-haley)
         Status: New


** Tags: ovn-octavia-provider

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1900763

Title:
  [OVN Octavia Provider] OVN provider status update failures can leave
  orphaned resources

Status in neutron:
  New

Bug description:
  This is related to Octavia issue
  https://storyboard.openstack.org/#!/story/2008254

  When the OVN Octavia Provider driver calls into octavia-lib to update
  the status of a loadbalancer, for example, the code in
  helper:_update_status_to_octavia() it might fail:

  DEBUG ovn_octavia_provider.helper [-] Updating status to octavia: {'listeners': [{'id': '7033179d-2ddb-4714-9c06-b7d399498238', 'provisioning_status>
  ERROR ovn_octavia_provider.helper [-] Error while updating the load balancer status: 'NoneType' object has no attribute 'update': octavia_lib.api.dr>
  ERROR ovn_octavia_provider.helper [-] Unexpected exception in request_handler: octavia_lib.api.drivers.exceptions.UpdateStatusError: ('The status up>
  ERROR ovn_octavia_provider.helper Traceback (most recent call last):
  ERROR ovn_octavia_provider.helper   File "/opt/stack/ovn-octavia-provider/ovn_octavia_provider/helper.py", line 318, in _update_status_to_octavia
  ERROR ovn_octavia_provider.helper     self._octavia_driver_lib.update_loadbalancer_status(status)
  ERROR ovn_octavia_provider.helper   File "/usr/local/lib/python3.8/dist-packages/octavia_lib/api/drivers/driver_lib.py", line 121, in update_loadbal>
  ERROR ovn_octavia_provider.helper     raise driver_exceptions.UpdateStatusError(
  ERROR ovn_octavia_provider.helper octavia_lib.api.drivers.exceptions.UpdateStatusError: 'NoneType' object has no attribute 'update'

  This is failing because the listener associated with the loadbalancer
  was not found, it's DB transaction was in-flight.  That's the related
  Octavia issue from above, but a fix for that will not solve the
  problem.

  A side-effect is this listener is now "stuck":

  $ openstack loadbalancer listener delete 7033179d-2ddb-4714-9c06-b7d399498238
  Load Balancer 2cc1d429-b176-48e5-adaa-946be2af0d51 is immutable and cannot be updated. (HTTP 409) (Request-ID: req-0e1e53ac-4db9-4779-b1f3-11210fe46f7f)

  The provider driver needs to retry the operation, typically even the
  very next call succeeds.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1900763/+subscriptions