← Back to team overview

graphite-dev team mailing list archive

Re: [Question #178316]: How will relay behave if the storage server(s) become unavailable?

 

Question #178316 on Graphite changed:
https://answers.launchpad.net/graphite/+question/178316

Abe Stanway posted a new comment:
@bhardy: After investigating, it only stops sending to all destinations
if USE_FLOW_CONTROL is on, as per chrismd. Otherwise, it sends to all
other destinations normally, and simply drops incoming datapoint for the
disconnected destination only.

We experienced issues in production using large MAX_QUEUE_VALUES and
dealing with disconnecting/reconnecting listeners. Specifically, when a
listener goes down, the queue fills up as it should, and relay stays
alive. However, when the listener reconnects, relay dies as it tries to
flush the queue all at once, because it blocks the IOLoop. Our solution
was to use a very small MAX_QUEUE_SIZE (5000).

There is a patch on the way from @mleinart to fix this behavior.

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.