yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #24952
[Bug 1396677] Re: Heavy use of metering labels/rules cause memory leak in neutron server
Even if this isn't a memory leak and not something easily fixable, the
behavior should at least be documented somewhere.
This may be relevent: http://effbot.org/pyfaq/why-doesnt-python-release-
the-memory-when-i-delete-a-large-object.htm
** Also affects: ceilometer
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1396677
Title:
Heavy use of metering labels/rules cause memory leak in neutron server
Status in OpenStack Telemetry (Ceilometer):
New
Status in OpenStack Neutron (virtual network service):
New
Bug description:
We found that large amount of metering labels and rules cause memory
leak in neutron server. This problem is multiplied by amount of
workers (10 workers - 10x memory leak).
In our case we have 657 metering-lables and 122399 metering-label-
rules,
If anyone query them, neutron-server (worker) picked request eats
+400Mb of memory and keep it until restart. If more requests send,
they come to different workers cause each of them to bloat up.
Same problem happens if neutron-plugin-metering-agent running (it send
requests to neutron-server with same effect).
If neutron-server hit 100% CPU it starts to consume even more memory
(in our case up to 1.4Gb per neutron-server worker).
To manage notifications about this bug go to:
https://bugs.launchpad.net/ceilometer/+bug/1396677/+subscriptions
References