← Back to team overview

graphite-dev team mailing list archive

Re: [Question #178921]: Limitation of Whisper sharding

 

Question #178921 on Graphite changed:
https://answers.launchpad.net/graphite/+question/178921

chrismd proposed the following answer:
Here's how it works for whisper. When the webapp gets a request to
render metric foo, it checks to see if foo exists locally and if it does
that's all it uses (plus a cache query but thats not really relevant).
If metric foo doesn't exist locally it broadcasts a 'find request' to
all webapps in the cluster (the results are memcached). Once it knows
which servers have metric foo, it requests the data from one webapp. The
webapp cannot effectively collate whisper data from disparate sources
because it would have to broadcast every request where one server might
have a missing datapoint, it has no way to know if the other servers
might have that datapoint. Caching only helps to a point, this would
lead to a lot of shared (as opposed to distributed) load.

Here's how it works for ceres. When the webapp gets a request to render
metric foo, it checks to see what intervals it has datapoints for metric
foo for. If it doesn't have all the data locally it broadcasts a find
request just as before but the results also include what intervals each
server has datapoints for each metric, not just which metrics it has.
Whisper doesn't make this data cheaply available, ceres does, and that
is really the main difference between the two.

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.