graphite-dev team mailing list archive

[David Gillies <question216675@xxxxxxxxxxxxxxxxxxxxx>]

New question #216675 on Graphite:
https://answers.launchpad.net/graphite/+question/216675

We've got the following setup on one server:

* 1x carbon-relay
* 3x carbon-caches behind the relay with consistent-hashing set

So when we start up the relay & 3 caches, memory usage isn't too bad and all the caches use approximately the same amount of memory and the cache size is roughly the same as well. But after a period of time, one or two (but not all) of the carbon-caches blow out in memory usage and the system starts tanking.

http://imgur.com/ZgGtT
http://imgur.com/Y8qbP

The weird part is that watching IO for each carbon-cache, it seems like at the moment, one of the caches is starving the other two for IO. Is there any way of tuning things so that one carbon-cache doesn't starve the others for IO?

Currently I've got:

MAX_UPDATES_PER_SECOND = 500
MAX_CREATES_PER_MINUTE = 400

Would tuning those values down help? How can I be sure that the IOs can be shared evenly? Is that at all possible?

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.