yahoo-eng-team team mailing list archive

[Rodolfo Alonso <2087953@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

References:
* https://etherpad.opendev.org/p/oct2024-ptg-neutron
* https://etherpad.opendev.org/p/neutron-eventlet-deprecation

Remove the import and usage of the eventlet library in the Neutron API.

============================================================

Worker execution.
=================
At this point we have solved the issue with ML2/OVN and the WSGI module. We have been migrating the U/S jobs from eventlet server to WSGI. That means the API threads do not use ``WorkerService``. ``WorkerService`` uses ``eventlet`` to spawn the server.

The periodic worker uses ``AllServicesNeutronWorker``, that uses
``oslo_service.service.Launcher``. This class uses ``eventlet`` pools
(called services)

The maintenance worker uses ``MaintenanceWorker`` and does not require
``eventlet``

Database access.
================
oslo.db implementation using asyncio: https://review.opendev.org/c/openstack/oslo.db/+/922976. This may or may not require some upgrade code in Neutron to migrate to the new config option. This will be assessed properly when the oslo.db implementation is merged.

However this new implementation is not strictly needed for the eventlet
deprecation process and we can test

Eventlet bits in the code.
==========================
There are several places in the Neutron API code where the ``eventlet`` library is called:
* https://github.com/openstack/neutron/blob/master/neutron/db/agents_db.py: used to introduce greenthread.sleep(0) to yield the thread. That could be removed. I don’t even think this is needed in the API because we don’t explicitly create multiple threads in an API worker.
* https://github.com/openstack/neutron/blob/master/neutron/notifiers/batch_notifier.py: could be refactored by using the ``threading`` Queue and sleep.
* https://github.com/openstack/neutron/blob/master/neutron/notifiers/nova.py: the threading implementation done in https://review.opendev.org/c/openstack/neutron/+/827331/2/neutron/notifiers/nova.py but with ``threading``.
* https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/plugin.py: waiting time in several port binding operations.
* https://github.com/openstack/neutron/blob/master/neutron/server/wsgi_eventlet.py: the entrypoint of the eventlet server (despite of the method name ``eventlet_wsgi_server``). It spawns a set of greenthreads waiting for the workers finalization.

RPC.
====
The same comment as in the agents. The RPC worker uses ``RpcWorker`` and does not require ``eventlet``. The RPC messaging engine (``oslo.messaging``) could work without ``eventlet``.

Neutron-lib patch to retrieve a server without defining the executor (it
will be taken from the context):
https://review.opendev.org/c/openstack/neutron-lib/+/930716

OVN IDL connections.
====================
Imports ``ovsdbapp`` to create IDL connections. This library and ``ovs`` (imported by ``ovsdbapp``) work with ``eventlet`` and normal threads.

** Affects: neutron
     Importance: Undecided
         Status: New

** Description changed:

  References:
  * https://etherpad.opendev.org/p/oct2024-ptg-neutron
  * https://etherpad.opendev.org/p/neutron-eventlet-deprecation
  
- Remove the import and usage of the eventlet library in the OVN metadata
- agent.
+ Remove the import and usage of the eventlet library in the Neutron API.
  
  ============================================================
  
  Worker execution.
  =================
  At this point we have solved the issue with ML2/OVN and the WSGI module. We have been migrating the U/S jobs from eventlet server to WSGI. That means the API threads do not use ``WorkerService``. ``WorkerService`` uses ``eventlet`` to spawn the server.
  
  The periodic worker uses ``AllServicesNeutronWorker``, that uses
  ``oslo_service.service.Launcher``. This class uses ``eventlet`` pools
  (called services)
  
  The maintenance worker uses ``MaintenanceWorker`` and does not require
  ``eventlet``
  
- 
  Database access.
  ================
  oslo.db implementation using asyncio: https://review.opendev.org/c/openstack/oslo.db/+/922976. This may or may not require some upgrade code in Neutron to migrate to the new config option. This will be assessed properly when the oslo.db implementation is merged.
  
  However this new implementation is not strictly needed for the eventlet
  deprecation process and we can test
- 
  
  Eventlet bits in the code.
  ==========================
  There are several places in the Neutron API code where the ``eventlet`` library is called:
  * https://github.com/openstack/neutron/blob/master/neutron/db/agents_db.py: used to introduce greenthread.sleep(0) to yield the thread. That could be removed. I don’t even think this is needed in the API because we don’t explicitly create multiple threads in an API worker.
  * https://github.com/openstack/neutron/blob/master/neutron/notifiers/batch_notifier.py: could be refactored by using the ``threading`` Queue and sleep.
  * https://github.com/openstack/neutron/blob/master/neutron/notifiers/nova.py: the threading implementation done in https://review.opendev.org/c/openstack/neutron/+/827331/2/neutron/notifiers/nova.py but with ``threading``.
  * https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/plugin.py: waiting time in several port binding operations.
  * https://github.com/openstack/neutron/blob/master/neutron/server/wsgi_eventlet.py: the entrypoint of the eventlet server (despite of the method name ``eventlet_wsgi_server``). It spawns a set of greenthreads waiting for the workers finalization.
  
- 
  RPC.
  ====
  The same comment as in the agents. The RPC worker uses ``RpcWorker`` and does not require ``eventlet``. The RPC messaging engine (``oslo.messaging``) could work without ``eventlet``.
  
  Neutron-lib patch to retrieve a server without defining the executor (it
  will be taken from the context):
  https://review.opendev.org/c/openstack/neutron-lib/+/930716
  
- 
  OVN IDL connections.
  ====================
  Imports ``ovsdbapp`` to create IDL connections. This library and ``ovs`` (imported by ``ovsdbapp``) work with ``eventlet`` and normal threads.

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2087953

Title:
  [eventlet-deprecation] Remove the usage of eventlet in the Neutron API

Status in neutron:
  New

Bug description:
  References:
  * https://etherpad.opendev.org/p/oct2024-ptg-neutron
  * https://etherpad.opendev.org/p/neutron-eventlet-deprecation

  Remove the import and usage of the eventlet library in the Neutron
  API.

  ============================================================

  Worker execution.
  =================
  At this point we have solved the issue with ML2/OVN and the WSGI module. We have been migrating the U/S jobs from eventlet server to WSGI. That means the API threads do not use ``WorkerService``. ``WorkerService`` uses ``eventlet`` to spawn the server.

  The periodic worker uses ``AllServicesNeutronWorker``, that uses
  ``oslo_service.service.Launcher``. This class uses ``eventlet`` pools
  (called services)

  The maintenance worker uses ``MaintenanceWorker`` and does not require
  ``eventlet``

  Database access.
  ================
  oslo.db implementation using asyncio: https://review.opendev.org/c/openstack/oslo.db/+/922976. This may or may not require some upgrade code in Neutron to migrate to the new config option. This will be assessed properly when the oslo.db implementation is merged.

  However this new implementation is not strictly needed for the
  eventlet deprecation process and we can test

  Eventlet bits in the code.
  ==========================
  There are several places in the Neutron API code where the ``eventlet`` library is called:
  * https://github.com/openstack/neutron/blob/master/neutron/db/agents_db.py: used to introduce greenthread.sleep(0) to yield the thread. That could be removed. I don’t even think this is needed in the API because we don’t explicitly create multiple threads in an API worker.
  * https://github.com/openstack/neutron/blob/master/neutron/notifiers/batch_notifier.py: could be refactored by using the ``threading`` Queue and sleep.
  * https://github.com/openstack/neutron/blob/master/neutron/notifiers/nova.py: the threading implementation done in https://review.opendev.org/c/openstack/neutron/+/827331/2/neutron/notifiers/nova.py but with ``threading``.
  * https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/plugin.py: waiting time in several port binding operations.
  * https://github.com/openstack/neutron/blob/master/neutron/server/wsgi_eventlet.py: the entrypoint of the eventlet server (despite of the method name ``eventlet_wsgi_server``). It spawns a set of greenthreads waiting for the workers finalization.

  RPC.
  ====
  The same comment as in the agents. The RPC worker uses ``RpcWorker`` and does not require ``eventlet``. The RPC messaging engine (``oslo.messaging``) could work without ``eventlet``.

  Neutron-lib patch to retrieve a server without defining the executor
  (it will be taken from the context):
  https://review.opendev.org/c/openstack/neutron-lib/+/930716

  OVN IDL connections.
  ====================
  Imports ``ovsdbapp`` to create IDL connections. This library and ``ovs`` (imported by ``ovsdbapp``) work with ``eventlet`` and normal threads.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2087953/+subscriptions