← Back to team overview

graphite-dev team mailing list archive

Re: [Question #285063]: Twisted MemoryError / MetricCache is full

 

Question #285063 on Graphite changed:
https://answers.launchpad.net/graphite/+question/285063

Denis Zhdanov posted a new comment:
1. Did you try carbon-c-relay? https://github.com/grobian/carbon-c-relay
It's written in C and has quite a good performance. Haproxy is not a real alternative, your carbon caches will clash with each other.
You need to use some hashing between caches and then only one cache will write specific metric, not all of them. 

2. Hope you mean 8MM metrics per _minute_ instead of second? I think
there's no way how you will be able to consume 8MM metrics/sec on a
single server, despite all tuning. IMO even 6 SSD in RAID10 will not
able to consume it. Maybe a couple of FusionIO's cards but I'm doubt.
You need a graphite cluster then, it's not that hard.

3. What you mean "values will arrive late to the system, sometimes up to
10-15m late"? Carbon cache has a cache (obviously) and graphite-web will
read data from cache first and flush data to disk with a quite big delay
- but that's not a problem usually. But for proper caching you need
matching order of your caches in carbon and graphite web config.

4. do you have monitoring for carbon metrics? all caches and relay write
it, could be useful for debugging.

-- 
You received this question notification because your team graphite-dev
is an answer contact for Graphite.