yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2065707] [NEW] Trunk plugin fails to push trunk object through OVO rpc endpoint on AFTER_DELETE

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Ihar Hrachyshka <2065707@xxxxxxxxxxxxxxxxxx>
Date: Tue, 14 May 2024 16:24:18 -0000
Reply-to: Bug 2065707 <2065707@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

>From here:
https://0ae702c1ad12e9119b7a-757cbada98b2345898c3deef0a647dfa.ssl.cf1.rackcdn.com/917263/3/check/neutron-
tempest-plugin-openvswitch/75602bd/controller/logs/screen-q-svc.txt

May 03 23:38:06.434640 np0037437196 neutron-server[58678]: DEBUG neutron.services.trunk.rpc.backend [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] Emitting event after_delete for resource trunk {{(pid=58678) process_event /opt/stack/neutron/neutron/services/trunk/rpc/backend.py:66}}
May 03 23:38:06.434711 np0037437196 neutron-server[58678]: DEBUG neutron.services.trunk.rpc.server [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] neutron.services.trunk.rpc.server.TrunkStub method trunk_deleted called with arguments (<neutron_lib.context.Context object at 0x7fc48ae428f0>, <neutron.db.models_v2.Port[object at 7fc48afc6e90] {project_id='d739e50798904e7582bbda9db4cb229b', id='94c147ce-6373-4d43-8916-5fba2e878ca5', name='', network_id='b6223fc1-458a-4b09-a024-7cfc9d960208', mac_address='fa:16:3e:0b:dc:8d', admin_state_up=True, status='DOWN', device_id='', device_owner='', ip_allocation='immediate', standard_attr_id=728}>) {} {{(pid=58678) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
May 03 23:38:06.434829 np0037437196 neutron-server[58678]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] Pushing event deleted for resources: {None: ['ID=94c147ce-6373-4d43-8916-5fba2e878ca5,revision_number=1']} {{(pid=58678) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}}
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] Error during notification for neutron.services.trunk.rpc.backend.ServerSideRpcBackend.process_event-504527 trunk, after_delete: neutron.api.rpc.handlers.resources_rpc.InvalidResourceTypeClass: Invalid resource type None
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager Traceback (most recent call last):
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/neutron-lib/neutron_lib/callbacks/manager.py", line 189, in _notify_loop
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     callback.method(resource, event, trigger, payload=payload)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/neutron/neutron/services/trunk/rpc/backend.py", line 67, in process_event
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     method[event](context, payload)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     return method(*args, **kwargs)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/neutron/neutron/services/trunk/rpc/server.py", line 208, in trunk_deleted
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     self._resource_rpc.push(context, [trunk], events.DELETED)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py", line 246, in push
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     self._push(context, resource_type, type_resources, event_type)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py", line 250, in _push
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     _validate_resource_type(resource_type)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py", line 52, in _validate_resource_type
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     raise InvalidResourceTypeClass(resource_type=resource_type)
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager neutron.api.rpc.handlers.resources_rpc.InvalidResourceTypeClass: Invalid resource type None
May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager 
May 03 23:38:06.437291 np0037437196 neutron-server[58678]: DEBUG neutron_lib.callbacks.manager [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] Publish callbacks [] for trunk (None), before_response {{(pid=58678) _notify_loop /opt/stack/neutron-lib/neutron_lib/callbacks/manager.py:184}}


I believe this was broken by https://review.opendev.org/c/openstack/neutron/+/885154

** Affects: neutron
     Importance: Undecided
     Assignee: Ihar Hrachyshka (ihar-hrachyshka)
         Status: In Progress

** Changed in: neutron
     Assignee: (unassigned) => Ihar Hrachyshka (ihar-hrachyshka)

** Changed in: neutron
       Status: New => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2065707

Title:
  Trunk plugin fails to push trunk object through OVO rpc endpoint on
  AFTER_DELETE

Status in neutron:
  In Progress

Bug description:
  From here:
  https://0ae702c1ad12e9119b7a-757cbada98b2345898c3deef0a647dfa.ssl.cf1.rackcdn.com/917263/3/check/neutron-
  tempest-plugin-openvswitch/75602bd/controller/logs/screen-q-svc.txt

  May 03 23:38:06.434640 np0037437196 neutron-server[58678]: DEBUG neutron.services.trunk.rpc.backend [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] Emitting event after_delete for resource trunk {{(pid=58678) process_event /opt/stack/neutron/neutron/services/trunk/rpc/backend.py:66}}
  May 03 23:38:06.434711 np0037437196 neutron-server[58678]: DEBUG neutron.services.trunk.rpc.server [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] neutron.services.trunk.rpc.server.TrunkStub method trunk_deleted called with arguments (<neutron_lib.context.Context object at 0x7fc48ae428f0>, <neutron.db.models_v2.Port[object at 7fc48afc6e90] {project_id='d739e50798904e7582bbda9db4cb229b', id='94c147ce-6373-4d43-8916-5fba2e878ca5', name='', network_id='b6223fc1-458a-4b09-a024-7cfc9d960208', mac_address='fa:16:3e:0b:dc:8d', admin_state_up=True, status='DOWN', device_id='', device_owner='', ip_allocation='immediate', standard_attr_id=728}>) {} {{(pid=58678) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
  May 03 23:38:06.434829 np0037437196 neutron-server[58678]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] Pushing event deleted for resources: {None: ['ID=94c147ce-6373-4d43-8916-5fba2e878ca5,revision_number=1']} {{(pid=58678) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}}
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] Error during notification for neutron.services.trunk.rpc.backend.ServerSideRpcBackend.process_event-504527 trunk, after_delete: neutron.api.rpc.handlers.resources_rpc.InvalidResourceTypeClass: Invalid resource type None
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager Traceback (most recent call last):
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/neutron-lib/neutron_lib/callbacks/manager.py", line 189, in _notify_loop
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     callback.method(resource, event, trigger, payload=payload)
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/neutron/neutron/services/trunk/rpc/backend.py", line 67, in process_event
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     method[event](context, payload)
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     return method(*args, **kwargs)
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/neutron/neutron/services/trunk/rpc/server.py", line 208, in trunk_deleted
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     self._resource_rpc.push(context, [trunk], events.DELETED)
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py", line 246, in push
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     self._push(context, resource_type, type_resources, event_type)
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py", line 250, in _push
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     _validate_resource_type(resource_type)
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager   File "/opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py", line 52, in _validate_resource_type
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager     raise InvalidResourceTypeClass(resource_type=resource_type)
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager neutron.api.rpc.handlers.resources_rpc.InvalidResourceTypeClass: Invalid resource type None
  May 03 23:38:06.435629 np0037437196 neutron-server[58678]: ERROR neutron_lib.callbacks.manager 
  May 03 23:38:06.437291 np0037437196 neutron-server[58678]: DEBUG neutron_lib.callbacks.manager [None req-343c488d-0f81-4741-bc0c-4e6da3883026 tempest-TagFilterTrunkTestJSON-125591612 tempest-TagFilterTrunkTestJSON-125591612-project-member] Publish callbacks [] for trunk (None), before_response {{(pid=58678) _notify_loop /opt/stack/neutron-lib/neutron_lib/callbacks/manager.py:184}}

  
  I believe this was broken by https://review.opendev.org/c/openstack/neutron/+/885154

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2065707/+subscriptions
Follow ups

[Bug 2065707] Re: Trunk plugin fails to push trunk object through OVO rpc endpoint on AFTER_DELETE
From: OpenStack Infra, 2024-06-02