graphite-dev team mailing list archive

[Cody Stevens <question176255@xxxxxxxxxxxxxxxxxxxxx>]

Question #176255 on Graphite changed:
https://answers.launchpad.net/graphite/+question/176255

    Status: Needs information => Open

Cody Stevens gave more information on the question:
I have only seen this issue with the carbon stats which ( I believe )
are handled internally to the carbon-cache.py daemon and would not rely
on the network at all.   Given that I did some research by pulling the
cache.size metrics with the rawData=True param and wrote a script to see
if this was occurring at a certain interval.  From what I found

Analyzing line for carbon.agents.graphch05.cache.size
Minutes since last occurrence: 190
Minutes since last occurrence: 47
Minutes since last occurrence: 334
Minutes since last occurrence: 142
Minutes since last occurrence: 47
Minutes since last occurrence: 94
Minutes since last occurrence: 47
Minutes since last occurrence: 238
Minutes since last occurrence: 47
Minutes since last occurrence: 95

It seems to be happening most frequently every 47 minutes... The other
numbers are roughly multiples of 47 minutes.

I have another script that calculates the average time between write cycles by analyzing the console.log.  
 	
It calculated a write cycle of  2876 seconds = 47.9333333 minutes

My theory is that the write of that metric data is occasionally
coinciding/colliding with another write somewhere and it is dropping the
data, thinking it got written.

I restarted carbon-cache.py on one server that was experiencing this
about 48 hours ago and the problem has not reoccurred on that server.

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.