← Back to team overview

graphite-dev team mailing list archive

Re: [Question #244564]: carbon-cache/whisper stopped writing wsp files over NFS share

 

Question #244564 on Graphite changed:
https://answers.launchpad.net/graphite/+question/244564

    Status: Open => Needs information

Travis Groth requested more information:
Bumping this thread.  We're encountering very similar behavior.  For no
apparent reason, carbon will stop writing data to disk, then start
increasing memory usage until either (a) we notice our graphs
slowing/stopping or (b) OOM killer takes it out.  We, however, are not
using NFS, not that it should matter much.

It is not a storage bottleneck - there is literally no IO happening.
We've been trying to determine if it is bad data coming in but we have
not had any luck in narrowing down what it might be or what would make
it "bad".  Sometimes we can go days without any problems, and sometimes
carbon won't run correctly for more than 20 minutes at a time.

There's absolutely zero information on what occurs in the logs.  Debug
doesn't seem to provide anything meaningful.  How can we troubleshoot
this further?

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.