← Back to team overview

graphite-dev team mailing list archive

[Question #228463]: In a cluster, how do queries travel?

 

New question #228463 on Graphite:
https://answers.launchpad.net/graphite/+question/228463

We have 3 Graphite servers, in a cluster.  2 of them use shared storage, that is, the /opt/graphite/storage mount points to the same data on both.  On the third, which has another source of data, it's a separate mount. (All three are, of course, on NFS mounted filers.)

Here is my question - while looking at the access log on the third system, I notice many queries from one of the other two for data in the shared storage area.  I thought that in a cluster, when a web server gets a request for data, it first looks at its own storage, and only if not found does it ask the other cluster members for data.  But in our case, the server that can "see" the data will ask the other server for data.  That server  cannot "see" the data locally.  It's confusing. 

I do not know which of the two shared-storage servers is running the carbon instance that writes the data to disk.  Does the web server determine whether to ask another cluster member for data by looking at its own storage directory, or does it somehow ask carbon what data is local?  Note that this question is complicated in our case as we run multiple carbon daemons, and some relays, in each server.

We're trying to track down a performance issue with the single storage Graphite web server, and noticed a LOT of these queries, which is what is prompting this question.

Thanks!
-Steve Keller

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.