yahoo-eng-team team mailing list archive

[Elena Ezhova <eezhova@xxxxxxxxxxxx>]

Public bug reported:

Neutron server has api and rpc workers and when their number is
configured to be non-zero each worker is launched using ProcessLauncher
from oslo-incubator's service.py. It is important to note that different
instances of ProcessLauncher are used  for launching api and rpc
workers. [1], [2]

When ProcessLauncher is initialized, among else it setups handlers for
termination signals (SIGHUP, SIGTERM and SIGINT) [3]. It is known that
only one signal handler can be installed per signal and only the latest
installed handler will be active. So, if several ProcessLauncher
instances are being initialized in the same process then only handlers
of the last instance will be triggered on receiving a signal.

The consequence is that when neutron-server is running with non-zero
number of api and rpc workers  sending a parent process SIGHUP would
result in reset method being called only for rpc workers.

The possible solution is to store all handlers in a class attribute and redefine handle_signal so that it calls all handlers one by one.
An alternative is to inherit from ProcessLauncher in neutron and redefine signal handling there.

[1] https://github.com/openstack/neutron/blob/e933891462408435c580ad42ff737f8bff428fbc/neutron/service.py#L159
[2] https://github.com/openstack/neutron/blob/e933891462408435c580ad42ff737f8bff428fbc/neutron/wsgi.py#L237
[3] https://github.com/openstack/oslo-incubator/blob/master/openstack/common/service.py#L210

** Affects: neutron
     Importance: Undecided
     Assignee: Elena Ezhova (eezhova)
         Status: Opinion

** Affects: oslo-incubator
     Importance: Undecided
     Assignee: Elena Ezhova (eezhova)
         Status: New

** Also affects: oslo-incubator
   Importance: Undecided
       Status: New

** Changed in: oslo-incubator
     Assignee: (unassigned) => Elena Ezhova (eezhova)

** Changed in: neutron
       Status: New => Opinion

** Changed in: neutron
     Assignee: (unassigned) => Elena Ezhova (eezhova)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1432995

Title:
  Termination signals not handled correctly in case of several
  ProcessLauncher instances in one process

Status in OpenStack Neutron (virtual network service):
  Opinion
Status in The Oslo library incubator:
  New

Bug description:
  Neutron server has api and rpc workers and when their number is
  configured to be non-zero each worker is launched using
  ProcessLauncher from oslo-incubator's service.py. It is important to
  note that different instances of ProcessLauncher are used  for
  launching api and rpc workers. [1], [2]

  When ProcessLauncher is initialized, among else it setups handlers for
  termination signals (SIGHUP, SIGTERM and SIGINT) [3]. It is known that
  only one signal handler can be installed per signal and only the
  latest installed handler will be active. So, if several
  ProcessLauncher instances are being initialized in the same process
  then only handlers of the last instance will be triggered on receiving
  a signal.

  The consequence is that when neutron-server is running with non-zero
  number of api and rpc workers  sending a parent process SIGHUP would
  result in reset method being called only for rpc workers.

  The possible solution is to store all handlers in a class attribute and redefine handle_signal so that it calls all handlers one by one.
  An alternative is to inherit from ProcessLauncher in neutron and redefine signal handling there.

  [1] https://github.com/openstack/neutron/blob/e933891462408435c580ad42ff737f8bff428fbc/neutron/service.py#L159
  [2] https://github.com/openstack/neutron/blob/e933891462408435c580ad42ff737f8bff428fbc/neutron/wsgi.py#L237
  [3] https://github.com/openstack/oslo-incubator/blob/master/openstack/common/service.py#L210

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1432995/+subscriptions