yahoo-eng-team team mailing list archive

[Salvatore Orlando <1420278@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

API workers are started with a fork().
It is well known that this operation uses CoW on the child process - and this should not constitute a problem.
However, the status synch thread is started at plugin initialization and might be already running when the API workers are forked.
The NSX API client,  extensively used by this thread, uses an eventlet semaphore to grab backend connections from a pool.
It is therefore possible that when a worker process is forked it receives semaphore which are in "busy" state. Once forked these semaphores are new objects, and they will never be unblocked. The API worker therefore simply hangs.

This behaviour has been confirmed by observation on the field

** Affects: neutron
     Importance: Undecided
         Status: Invalid

** Affects: neutron/icehouse
     Importance: High
     Assignee: Salvatore Orlando (salvatore-orlando)
         Status: New

** Affects: neutron/juno
     Importance: High
     Assignee: Salvatore Orlando (salvatore-orlando)
         Status: New

** Affects: vmware-nsx
     Importance: High
     Assignee: Salvatore Orlando (salvatore-orlando)
         Status: In Progress

** Also affects: neutron
   Importance: Undecided
       Status: New

** Also affects: neutron/icehouse
   Importance: Undecided
       Status: New

** Also affects: neutron/juno
   Importance: Undecided
       Status: New

** Changed in: neutron/juno
     Assignee: (unassigned) => Salvatore Orlando (salvatore-orlando)

** Changed in: neutron/icehouse
     Assignee: (unassigned) => Salvatore Orlando (salvatore-orlando)

** Changed in: neutron
       Status: New => Invalid

** Changed in: neutron/juno
   Importance: Undecided => High

** Changed in: neutron/icehouse
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1420278

Title:
  API workers might not work when sync thread is enabled

Status in OpenStack Neutron (virtual network service):
  Invalid
Status in neutron icehouse series:
  New
Status in neutron juno series:
  New
Status in VMware NSX:
  In Progress

Bug description:
  API workers are started with a fork().
  It is well known that this operation uses CoW on the child process - and this should not constitute a problem.
  However, the status synch thread is started at plugin initialization and might be already running when the API workers are forked.
  The NSX API client,  extensively used by this thread, uses an eventlet semaphore to grab backend connections from a pool.
  It is therefore possible that when a worker process is forked it receives semaphore which are in "busy" state. Once forked these semaphores are new objects, and they will never be unblocked. The API worker therefore simply hangs.

  This behaviour has been confirmed by observation on the field

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1420278/+subscriptions