graphite-dev team mailing list archive

[Dmitry <question267948@xxxxxxxxxxxxxxxxxxxxx>]

New question #267948 on Graphite:
https://answers.launchpad.net/graphite/+question/267948

Hi there,

I'm at the point, when I decided to ask for help from the Graphite community. :)
The problem I'm facing is a rather strange loss of data in one of two graphite setups I'm sending stats data to.
My setup is pretty simple and look like this:

data-source -> carbon-aggregator -> carbon-relay -> carbon-cache

Where carbon-aggregator is configured with FORWARD_ALL option enabled and passes original and aggregated data further to carbon-relay which is configured to send the original (not aggregated data) to local carbon-cache, the aggregates go to the same local carbon-cache, plus to another remote graphite setup which I call central graphite.

All graphs of aggregated metrics from the local graphite look good, but the same graphs from the central graphite have quite a few sudden gaps/drops which I can not explain.
To better explain what I mean, please have a look at these images: http://imgur.com/a/g4nh5
* First image shows two graphs from the local graphite.
* Second image shows the same graphs, for the same period of time from the central graphite server.

I was trying to dig into the problem but the only strange thing I spotted was aggregateDatapointsSent graph for local carbon-aggregator daemon, you can see it on the third image in the centre. Oddly enough the ups and downs on this graph correlate with downs and ups on the application metric graphs from the central graphite. Maybe it's not related at all but looks suspicious.

One more thing to mention is that we run different versions of graphite for the local and central setups. Local setup has graphite 0.9.13 installed while the central one - 0.9.12.

Other than that I don't really know what to put the blame on. Any suggestions/ideas would be very much appreciated.
I can also provide more details on my setup if needed.

Thanks,
Dmitry

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.