yahoo-eng-team team mailing list archive

[Rodolfo Alonso <2070376@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

The improvement done in [1] helps to process the RPC events, assigning a
priority to these events and optimizing the dispatch strategy. However
the multithreading processing of these events introduced in this patch
does not add any performance improvement. The DHCP agent is a single
process service and due to the nature of CPython, there is no real
multithreading during the execution of Python code.

We are now currently using green threads (``eventlet.GreenPool``) to spawn the resource update method (the method that reads the ordered list of events and performs the needed action in the DHCP driver). Any action is stored in a priority queue and the first available resource update thread will process it. However this implementation has several drawbacks:
* It doesn't provide any performance improvement. Having multiple threads running Python code doesn't outperform a single thread process.
* If we switch from collaborative threads (green threads) to pre-emptive threads (kernel threads), we must implement locking blocks for the resource update methods, that will also result in more restricted thread executions.

This bug proposes to remove the resource update threading and leave a
single thread that will read the event queue and perform the needed
actions.

[1]https://review.opendev.org/c/openstack/neutron/+/626830

** Affects: neutron
     Importance: Undecided
     Assignee: Rodolfo Alonso (rodolfo-alonso-hernandez)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Rodolfo Alonso (rodolfo-alonso-hernandez)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2070376

Title:
  [eventlet-deprecation] Reduce the DHCP agent pool processing to a
  single thread

Status in neutron:
  New

Bug description:
  The improvement done in [1] helps to process the RPC events, assigning
  a priority to these events and optimizing the dispatch strategy.
  However the multithreading processing of these events introduced in
  this patch does not add any performance improvement. The DHCP agent is
  a single process service and due to the nature of CPython, there is no
  real multithreading during the execution of Python code.

  We are now currently using green threads (``eventlet.GreenPool``) to spawn the resource update method (the method that reads the ordered list of events and performs the needed action in the DHCP driver). Any action is stored in a priority queue and the first available resource update thread will process it. However this implementation has several drawbacks:
  * It doesn't provide any performance improvement. Having multiple threads running Python code doesn't outperform a single thread process.
  * If we switch from collaborative threads (green threads) to pre-emptive threads (kernel threads), we must implement locking blocks for the resource update methods, that will also result in more restricted thread executions.

  This bug proposes to remove the resource update threading and leave a
  single thread that will read the event queue and perform the needed
  actions.

  [1]https://review.opendev.org/c/openstack/neutron/+/626830

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2070376/+subscriptions