graphite-dev team mailing list archive

[Michael Leinartas <question198435@xxxxxxxxxxxxxxxxxxxxx>]

Question #198435 on Graphite changed:
https://answers.launchpad.net/graphite/+question/198435

    Status: Open => Answered

Michael Leinartas proposed the following answer:
The biggest concern in this case would be how well your filesystem
handles the metric structure you decide on. Whisper files are stored in
a directory structure that maps to the metric name - i.e.
metrics.server1.cpu.idle would map to metrics/server1/cpu/idle.wsp. In
the case of millions of metrics, this means millions of files. Likely
you'll want to ensure that your metric structure isn't too flat so that
you dont have hundreds of thousands or millions of files in a single
directory. Graphite's metric browser opens these metric directories as
the user browses so a properly branched structure would reduce both the
time it takes to read the directory listing on disk at each node
expansion as well as reduce the time to transfer and render the data in
the browser.

The other thing to think of is how you'll be rendering this data - where
you may run into trouble is if you plan to render a great many of these
(say 50-100 thousand or more) in a single graph using wildcards. In that
case the time it takes to find the metrics, read the data for the period
requested, and apply any functions may take quite some time and memory.
Graphite does provide a means to cache rendered graphs and requested
data in memcached so it could still be feasible to draw graphs like this
with aggressive caching if your use-case fits.

Make sure you let us know how it bakes up!

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.