yahoo-eng-team team mailing list archive

[Thomas Morin <1750777@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

We just ran into a case where the openvswitch agent (local dev destack,
current master branch) eats 100% of CPU time.

Pyflame profiling show the time being largely spent in
neutron.agent.linux.ip_conntrack, line 95.

https://github.com/openstack/neutron/blob/master/neutron/agent/linux/ip_conntrack.py#L95

The code around this line is:

        while True:
            pool.spawn_n(self._process_queue)

The documentation of eventlet.spawn_n says: "The same as spawn(), but
it’s not possible to know how the function terminated (i.e. no return
value or exceptions). This makes execution faster. See spawn_n for more
details."  I suspect that GreenPool.spaw_n may behave similarly.

It seems plausible that spawn_n is returning very quickly because of
some error, and then all time is quickly spent in a short circuited
while loop.

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1750777

Title:
  openvswitch agent eating CPU, time spent in ip_conntrack.py

Status in neutron:
  New

Bug description:
  We just ran into a case where the openvswitch agent (local dev
  destack, current master branch) eats 100% of CPU time.

  Pyflame profiling show the time being largely spent in
  neutron.agent.linux.ip_conntrack, line 95.

  https://github.com/openstack/neutron/blob/master/neutron/agent/linux/ip_conntrack.py#L95

  The code around this line is:

          while True:
              pool.spawn_n(self._process_queue)

  The documentation of eventlet.spawn_n says: "The same as spawn(), but
  it’s not possible to know how the function terminated (i.e. no return
  value or exceptions). This makes execution faster. See spawn_n for
  more details."  I suspect that GreenPool.spaw_n may behave similarly.

  It seems plausible that spawn_n is returning very quickly because of
  some error, and then all time is quickly spent in a short circuited
  while loop.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1750777/+subscriptions