graphite-dev team mailing list archive

[Yee-Ting Li <question214631@xxxxxxxxxxxxxxxxxxxxx>]

Question #214631 on Graphite changed:
https://answers.launchpad.net/graphite/+question/214631

Yee-Ting Li posted a new comment:
5,000: trail and error i'm afraid. i guess the systematic way would be
to run a bonnie++ test; however, with the read and write YMMV - but with
linux kernel caching etc, it gets messy. i would like to say that i'm
trying to keep the IOWAIT to around 20%, but that was just a happy
coincidence ;)

wrt WHISPER_AUTOFLUSH = True, i haven't tried it but it does sound
interesting. please tell me if you have any good experiences with it
(i'm rather nervous about prodding a stable system)

FLOW_CONTROL: so i would use the carbon-relay to provide a single point
of entry into carbon, however, i noticed that it gets rather cpu bound
with a large number of metrics. so i basically wrote a simple process
that spawns off n processes (let's call them feeders); each process
connects to a single carbon-cache (so 16 carbon-caches = 16 feeders).
within my code that feeds into carbon, i precompute which carbon-cache
would be hit using consistent hashing and then send it just to the
relevant feeder. i find that i am able to push >1million metrics/minute
using this method (whether carbon can actually consume it is another
question). therefore, if the underlying carbon-instance hits it's cache
limit, the feeder's socket to that instance is closed, i notice in the
feeder and start buffering.

so the amount of time it takes to appear depends on what you mean. i
believe the issue is that the webapp requires that the whisper file
exists on the disk for it to show up in the side bar. therefore, with
metrics that have never been seen before, it can take a long time
(MAX_CREATES). however, if you already know the name of the metric, the
webapp /should/ calculate the relevant carbon-instance to query it's
cache so it should be there as soon as an instance receives it. the best
way to check this is to look at log/webapp/cache.log.

no probs with sharing my experiences: i find that tuning carbon for high
performance is really quite a pain.

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.