yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #72701
[Bug 1750777] Re: openvswitch agent eating CPU, time spent in ip_conntrack.py
** Also affects: neutron (Ubuntu)
Importance: Undecided
Status: New
** Changed in: neutron (Ubuntu)
Status: New => Triaged
** Changed in: neutron (Ubuntu)
Importance: Undecided => High
** Also affects: neutron (Ubuntu Bionic)
Importance: Undecided
Status: New
** Also affects: neutron (Ubuntu Cosmic)
Importance: High
Status: Triaged
** Also affects: cloud-archive
Importance: Undecided
Status: New
** Also affects: cloud-archive/queens
Importance: Undecided
Status: New
** Changed in: cloud-archive/queens
Status: New => Triaged
** Changed in: neutron (Ubuntu Bionic)
Importance: Undecided => High
** Changed in: neutron (Ubuntu Bionic)
Status: New => Triaged
** Changed in: cloud-archive/queens
Importance: Undecided => High
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1750777
Title:
openvswitch agent eating CPU, time spent in ip_conntrack.py
Status in Ubuntu Cloud Archive:
Triaged
Status in Ubuntu Cloud Archive queens series:
Triaged
Status in neutron:
Fix Released
Status in neutron package in Ubuntu:
Triaged
Status in neutron source package in Bionic:
Triaged
Status in neutron source package in Cosmic:
Triaged
Bug description:
We just ran into a case where the openvswitch agent (local dev
destack, current master branch) eats 100% of CPU time.
Pyflame profiling show the time being largely spent in
neutron.agent.linux.ip_conntrack, line 95.
https://github.com/openstack/neutron/blob/master/neutron/agent/linux/ip_conntrack.py#L95
The code around this line is:
while True:
pool.spawn_n(self._process_queue)
The documentation of eventlet.spawn_n says: "The same as spawn(), but
it’s not possible to know how the function terminated (i.e. no return
value or exceptions). This makes execution faster. See spawn_n for
more details." I suspect that GreenPool.spawn_n may behave similarly.
It seems plausible that spawn_n is returning very quickly because of
some error, and then all time is quickly spent in a short circuited
while loop.
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1750777/+subscriptions
References