yahoo-eng-team team mailing list archive

[OpenStack Infra <2065701@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/neutron/+/919699
Committed: https://opendev.org/openstack/neutron/commit/79676270354112fd4062442d25a5eaefbc830d24
Submitter: "Zuul (22348)"
Branch:    master

commit 79676270354112fd4062442d25a5eaefbc830d24
Author: yatinkarel <ykarel@xxxxxxxxxx>
Date:   Wed May 15 14:16:53 2024 +0530

    Do not fail on missing logical router ports
    
    set_gateway_mtu runs for all the gateway ports for a network
    and if one of the ports get's deleted in meanwhile
    whole transaction fails. Added if_exists=True so the
    transaction do not fail for such deleted ports.
    
    Also update minimal version of "ovsdbapp" required for
    this.
    
    Depends-On: https://review.opendev.org/c/openstack/ovsdbapp/+/919696
    Depends-On: https://review.opendev.org/c/openstack/requirements/+/920062
    Closes-Bug: #2065701
    Related-Bug: #2060163
    Change-Id: Ie0ce420414f1e6056d9df2733b7446d02c89fcfc


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2065701

Title:
  set_gateway_mtu fails if one of the lrp is deleted while it's running

Status in neutron:
  Fix Released

Bug description:
  Seen in some downstream jobs running with antelope content and
  ovn_emit_need_to_frag = true, tests fails randomly while running
  add_router_interface or remove_router_interface calls like:-

  ft9.1: setUpClass (tempest.api.compute.servers.test_server_actions.ServerActionsV293TestJSON)testtools.testresult.real._StringException: Traceback (most recent call last):
    File "/usr/lib/python3.9/site-packages/tempest/test.py", line 206, in setUpClass
      raise value.with_traceback(trace)
    File "/usr/lib/python3.9/site-packages/tempest/test.py", line 191, in setUpClass
      cls.setup_credentials()
    File "/usr/lib/python3.9/site-packages/tempest/api/compute/servers/test_server_actions.py", line 827, in setup_credentials
      super(ServerActionsV293TestJSON, cls).setup_credentials()
    File "/usr/lib/python3.9/site-packages/tempest/api/compute/base.py", line 75, in setup_credentials
      super(BaseV2ComputeTest, cls).setup_credentials()
    File "/usr/lib/python3.9/site-packages/tempest/test.py", line 419, in setup_credentials
      manager = cls.get_client_manager(
    File "/usr/lib/python3.9/site-packages/tempest/test.py", line 764, in get_client_manager
      creds = getattr(cred_provider, credentials_method)()
    File "/usr/lib/python3.9/site-packages/tempest/lib/common/dynamic_creds.py", line 473, in get_primary_creds
      return self.get_project_member_creds()
    File "/usr/lib/python3.9/site-packages/tempest/lib/common/dynamic_creds.py", line 508, in get_project_member_creds
      return self.get_credentials(['member'], scope='project')
    File "/usr/lib/python3.9/site-packages/tempest/lib/common/dynamic_creds.py", line 459, in get_credentials
      network, subnet, router = self._create_network_resources(
    File "/usr/lib/python3.9/site-packages/tempest/lib/common/dynamic_creds.py", line 319, in _create_network_resources
      self._add_router_interface(router['id'], subnet['id'])
    File "/usr/lib/python3.9/site-packages/tempest/lib/common/dynamic_creds.py", line 383, in _add_router_interface
      self.routers_admin_client.add_router_interface(router_id,
    File "/usr/lib/python3.9/site-packages/tempest/lib/services/network/routers_client.py", line 72, in add_router_interface
      return self.update_resource(uri, kwargs)
    File "/usr/lib/python3.9/site-packages/tempest/lib/services/network/base.py", line 77, in update_resource
      resp, body = self.put(req_uri, req_post_data)
    File "/usr/lib/python3.9/site-packages/tempest/lib/common/rest_client.py", line 372, in put
      return self.request('PUT', url, extra_headers, headers, body, chunked)
    File "/usr/lib/python3.9/site-packages/tempest/lib/common/rest_client.py", line 742, in request
      self._error_checker(resp, resp_body)
    File "/usr/lib/python3.9/site-packages/tempest/lib/common/rest_client.py", line 922, in _error_checker
      raise exceptions.ServerFault(resp_body, resp=resp,
  tempest.lib.exceptions.ServerFault: Got server fault
  Details: Request Failed: internal server error while processing your request.

  Fails with Internal server error:-
  192.168.16.2 - - [13/May/2024:15:09:44 +0000] "PUT /v2.0/routers/a76e7487-6fcb-4134-8de4-f08a3eca33cb/add_router_interface HTTP/1.1" 500 150 "-" "python-urllib3/1.26.5"

  2024-05-13T15:10:15.685665853+00:00 stdout F 2024-05-13 15:10:15.684 16 DEBUG ovsdbapp.backend.ovs_idl.transaction [None req-8b7c93b4-c0d8-4963-b711-cab362fcada4 - - - - - -] Running txn n=1 command(idx=20): LrpSetOptionsCommand(_result=None, entity=lrp-9309bd6d-2e46-404e-8c47-fda1adc15cc5, options={}) do_commit /usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89^[[00m
  2024-05-13T15:10:15.686489887+00:00 stdout F 2024-05-13 15:10:15.685 16 ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d211c9ca-9694-4d69-9e57-b330a231c30f 9486b7f231584e7ab8f143c840957bf3 6bcd5aaa226d4e69b9e20287ea2d3b11 - - default default] Traceback (most recent call last):
  2024-05-13T15:10:15.686489887+00:00 stdout F   File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 118, in run
  2024-05-13T15:10:15.686489887+00:00 stdout F     txn.results.put(txn.do_commit())
  2024-05-13T15:10:15.686489887+00:00 stdout F   File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 92, in do_commit
  2024-05-13T15:10:15.686489887+00:00 stdout F     command.run_idl(txn)
  2024-05-13T15:10:15.686489887+00:00 stdout F   File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 349, in run_idl
  2024-05-13T15:10:15.686489887+00:00 stdout F     entity = self.api.lookup(self.table, self.entity)
  2024-05-13T15:10:15.686489887+00:00 stdout F   File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 183, in lookup
  2024-05-13T15:10:15.686489887+00:00 stdout F     return self._lookup(table, record)
  2024-05-13T15:10:15.686489887+00:00 stdout F   File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 234, in _lookup
  2024-05-13T15:10:15.686489887+00:00 stdout F     row = idlutils.row_by_value(self, rl.table, rl.column, record)
  2024-05-13T15:10:15.686489887+00:00 stdout F   File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
  2024-05-13T15:10:15.686489887+00:00 stdout F     raise RowNotFound(table=table, col=column, match=match)
  2024-05-13T15:10:15.686489887+00:00 stdout F ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router_Port with name=lrp-9309bd6d-2e46-404e-8c47-fda1adc15cc5
  2024-05-13T15:10:15.686489887+00:00 stdout F ^[[00m

  
  This happens as router port is removed while the LrpSetOptionsCommand runs for that router port, the transaction assumes the port exists[1]. Would need to add if-exists=true in LrpSetOptionsCommand/lrp_set_options to handle such cases.

  [1]
  https://opendev.org/openstack/neutron/src/branch/master/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py#L2084-L2088

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2065701/+subscriptions