graphite-dev team mailing list archive

Thread
Date

[Question #241496]: Memory Usage/issues with Graphite

To: graphite-dev@xxxxxxxxxxxxxxxxxxx
From: Gonzalo <question241496@xxxxxxxxxxxxxxxxxxxxx>
Date: Tue, 31 Dec 2013 15:51:07 -0000
Reply-to: question241496@xxxxxxxxxxxxxxxxxxxxx
Sender: bounces@xxxxxxxxxxxxx

New question #241496 on Graphite:
https://answers.launchpad.net/graphite/+question/241496

Hi

I have already spent many days on this I can only see how the memory vanishes in a few hours and eventually I cannot use the Web GUI as it does not longer work given carbon has literally eaten all the system resources..

So now I basically got 3 cache servers with the following configuration:

------------

[cache]
USER =

LINE_RECEIVER_INTERFACE = 0.0.0.0
PICKLE_RECEIVER_INTERFACE = 0.0.0.0
CACHE_QUERY_INTERFACE = 0.0.0.0

[cache:a]
MAX_CACHE_SIZE = 10000000
MAX_UPDATES_PER_SECOND = 1000
MAX_CREATES_PER_MINUTE = 50

LINE_RECEIVER_PORT = 2013
PICKLE_RECEIVER_PORT = 2014
CACHE_QUERY_PORT = 7101

USE_INSECURE_UNPICKLER = False
USE_FLOW_CONTROL = False
LOG_UPDATES = False
WHISPER_AUTOFLUSH = False
MAX_UPDATES_PER_SECOND_ON_SHUTDOWN = 1500


# To configure special settings for the carbon-cache instance 'b', uncomment this:
[cache:b]
MAX_CACHE_SIZE = 10000000
MAX_UPDATES_PER_SECOND = 1000
MAX_CREATES_PER_MINUTE = 50

LINE_RECEIVER_PORT = 2023
PICKLE_RECEIVER_PORT = 2024
CACHE_QUERY_PORT = 7102

USE_INSECURE_UNPICKLER = False
USE_FLOW_CONTROL = False
LOG_UPDATES = False
WHISPER_AUTOFLUSH = False
MAX_UPDATES_PER_SECOND_ON_SHUTDOWN = 1500


[cache:c]
MAX_CACHE_SIZE = 10000000
MAX_UPDATES_PER_SECOND = 1000
MAX_CREATES_PER_MINUTE = 50

LINE_RECEIVER_PORT = 2033
PICKLE_RECEIVER_PORT = 2034
CACHE_QUERY_PORT = 7103

USE_INSECURE_UNPICKLER = False
USE_FLOW_CONTROL = False
LOG_UPDATES = False
WHISPER_AUTOFLUSH = False
MAX_UPDATES_PER_SECOND_ON_SHUTDOWN = 1500

-------------

This is a dedicated machine with 32 Gbs of RAM.

For any chance, is it possible to actually limit the memory usage without stopping the connections at all?
If I limit the Cache_size and I got USE_FLOW_CONTROL as True it will stop getting more data until some resources get free and if Unlimited it will eventually exhaust the system resources..

iostats for that device as follows:

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
dm-0           1344.95       766.45     10407.15 3266714538 44356963504


Any Tips?

Thanks very much


-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.