yahoo-eng-team team mailing list archive

[OpenStack Infra <2077790@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/neutron/+/926922
Committed: https://opendev.org/openstack/neutron/commit/ae90e2ccbfa45a8e864ec6f7fca2f28fa90d8062
Submitter: "Zuul (22348)"
Branch:    master

commit ae90e2ccbfa45a8e864ec6f7fca2f28fa90d8062
Author: Rodolfo Alonso Hernandez <ralonsoh@xxxxxxxxxx>
Date:   Sat Aug 24 10:35:03 2024 +0000

    Make RPC event cast synchronous with the event
    
    Sometimes, the methods ``NeutronObject.get_object`` and
    ``ResourcesPushRpcApi.push`` yield the GIL during the execution.
    Because of that, the thread in charge of sending the RPC information
    doesn't finish until other operation is pushed (implemented in [1]).
    
    By making the RPC cast synchronous with the update/delete events, it
    is ensured that both operations will finish and the agents will receive
    the RPC event on time, just after the event happens.
    
    This issue is hitting more frequently in the migration to the WSGI
    server, due to [2]. Once the eventlet library has been deprecated from
    OpenStack, it will be possible to use the previous model (using a long
    thread to handle the RCP updates to the agents). It is commented in the
    code as a TODO.
    
    This patch is temporarily reverting [3]. This code should be restored
    too.
    
    [1]https://review.opendev.org/c/openstack/neutron/+/788510
    [2]https://review.opendev.org/c/openstack/neutron/+/925376
    [3]https://review.opendev.org/c/openstack/neutron/+/824508
    
    Closes-Bug: #2077790
    Related-Bug: #2075147
    Change-Id: I7b806e6de74164ad9730480a115a76d30e7f15fc


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2077790

Title:
  [eventlet] RPC handler thread model is incompatible with eventlet

Status in neutron:
  Fix Released

Bug description:
  The RPC handler class ``_ObjectChangeHandler``, that is instantiated
  in ``OVOServerRpcInterface``, is not eventlet compatible.

  The ``OVOServerRpcInterface`` class is in charge of receiving the
  resource events (port, network, SG, etc.) and send this update via RPC
  to the listeners (agents like OVS agent or DHCP agent). Since [1], we
  create a single long running thread that reads the stored events and
  sends the RPC message (``RPCClient.cast`` because it is not expected a
  reply).

  Although this architecture is correct, it is not fully compatible with
  eventlet. Since [2] and the upper patches testing this patch, the OVS
  jobs (that use RPC between the server and the agents) are randomly
  failing. This is more frequently with the SG API operations (SG rule
  addition and deletion).

  This bug proposes to make the event RPC cast synchronous with the API
  call, avoiding using a thread to collect and send the RPC messages.
  Once eventlet is removed from the OpenStack project, we'll be able to
  use the previous model.

  POC patch: https://review.opendev.org/c/openstack/neutron/+/926922
  Testing patch: https://review.opendev.org/c/openstack/neutron/+/926788

  [1]https://review.opendev.org/c/openstack/neutron/+/788510
  [2]https://review.opendev.org/c/openstack/neutron/+/925376

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2077790/+subscriptions