← Back to team overview

graphite-dev team mailing list archive

Re: [Question #177524]: can graphie scale to Ks of nodes at frequencies of seconds

 

Question #177524 on Graphite changed:
https://answers.launchpad.net/graphite/+question/177524

Mark Seger gave more information on the question:
maybe I should have looked at the API before I posted this as it does
look very simple...  ;)

I'm not a python person  but it looks like you simply connect to a socket and do a print, right?
Turns out collectl already works like that as well AND has something real close to this model.  When it starts up you give it an address/port and it too does a print to that socket.  In collectl's case the data looks like this (I chose loadavg because that's what's in the example ;)):

      $cpuSumString.=sendData("cpuload.avg1",  $loadAvg1, '%4.2f');
      $cpuSumString.=sendData("cpuload.avg5",  $loadAvg5, '%4.2f');
      $cpuSumString.=sendData("cpuload.avg15", $loadAvg15,'%4.2f');

which turns into this:

sample.time 1320411891.002
cpuload.avg1 0.00
cpuload.avg5 0.00
cpuload.avg15 0.04

as you can see, collectl reports the time as a separate line, but it
would be trivial to change.

one thing that is not clear from the example is how to deal with error
handling.  in the case of collectl, it has two models of use.  In the
one I prefer, it currently expects the other end to connect to it and
accepts the connection.  Then, every monitoring interval collect does a
send.  If the send fails, collectl closes the socket and goes back to
listening for a connection.  I believe it should never be forced to exit
since it can be simultaneously logging more detailed data to a local
file and just because the other end went away is no reason to stop
logging.

collectl also has the ability to connect to an existing socket, but the
way the code is currently written, when that connection goes away
collectl will exit and I consider that a bad thing for long-term
monitoring.

there is also a special mechanism I wrote that allows collectl to write
over a UDP socket directly to a gangila gmond.  this solution is running
on a 2300 node cluster where ganglia is only used to distribute the
data, not write it to rrd.

-mark

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.