graphite-dev team mailing list archive

[chrismd <question83274@xxxxxxxxxxxxxxxxxxxxx>]

Question #83274 on Graphite changed:
https://answers.launchpad.net/graphite/+question/83274

chrismd posted a new comment:
Glad to help Steve but I'd like to offer a tiny bit more advice if thats
alright. 20 minutes is obviously way too long for a request like this to
take, and I don't even know the specifics, it just *sounds* bad! So I'll
offer up some thoughts on what might be causing this.

So if you have 3,048 different metrics involved in your request, then
yes that will correspond to 3,048 different whisper files. If your data
is stored with minutely precision and you're using from=-1d then you are
extracting 1,440 data points per metric (one for each minute in the past
24 hours). Each data point is 12 bytes on disk, so your entire request
requires graphite to read approximately 50 megs of data off the disk (12
bytes * 1440 * 3048). That's not a huge amount but keep in mind that
since it is spread across 3,048 different files your disk will be doing
a lot of seeking so that may be the cause of the problem. Again this
estimate assumes your data is stored with minutely precision which may
not be the case, but in any case I think the following recommendation
may help.

I assume this type of request is a common thing you will need to do
(otherwise waiting 20 minutes probably wouldn't be a big deal). As a
general rule of thumb I try to optimize the data I send to graphite for
its common use cases. For example, a long time ago I had about 100 or so
performance metrics that I almost always viewed in a single aggregate
graph and seldom in isolation. So to make my big graph render quicker I
started pre-calculating the aggregate values it displayed and sent those
directly to graphite in addition to the individual metrics. This meant
that when I requested the big aggregate graph only a single database
file was involved. Granted this particular technique may not apply in
your use case, but I would bet that some variant of it could help. 3,048
different metrics (multipled by however many datapoints you store for
each per day) is a lot of data points so I assume you are doing some
sort of post-processing on this data to summarize it after you retrieve
it. My recommendation would be to try and think of a way to send some
sort of pre-calculated values into graphite that would make your
computation involve less data points later on. I hope that helps.

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.