← Back to team overview

graphite-dev team mailing list archive

[Question #263285]: carbon-aggregator latest partial datapoint

 

New question #263285 on Graphite:
https://answers.launchpad.net/graphite/+question/263285

Hi, 

I have a SSD based node with carbon-aggregator being in front of 5 carbon-caches. I have a problem with aggregated metrics, the latest value that is stored in aggregated whisper files is always +/- twice lower than the usual metric value.

In the following case im aggregating data from 80 nodes (cpu usage). 

Whisper file for cpu-idle looks like this:

1425610260      99681.524708
1425610320      99755.657654
1425610380      51996.786829
1425610440      None

As you can see the latest not-none value is smaller than first 2 values (which are regular ones). After next datapoint come in:

1425610260      99681.524708
1425610320      99755.657654
1425610380      100871.700813
1425610440      52114.747810
1425610500      None

As you can see the latest value is again twice smaller. This becomes very confusing with low resolution data because there is always a fake-drop, and when the real drop appears we wont notice it and wont be able to react fast.

Im feeding graphite with collectd, with interval of 60 seconds, the aggregation rule looks like:

  * Aggregated.hosting.cpu.<cpumetric> (60) = sum Hosting.node.*.cpu-*.<cpumetric>

And my config: http://pastebin.com/Ljtrwnm7

Playing with MAX_UPDATES_PER_SECOND didnt help me. Tried also to play with the aggregation method buffering time but it didnt make the trick. When i compse graph of using aggregated data then it is looking like at the end there is a big drop (a half value). However, when I compose this graph using non-aggregated data (just aggregating on client side in grafana with queries with "*") the graph is looking alright (no drops). 

What can cause this behaviour? 


-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.