← Back to team overview

graphite-dev team mailing list archive

Re: [Question #173171]: carbon-relay seems to be broken

 

Question #173171 on Graphite changed:
https://answers.launchpad.net/graphite/+question/173171

Description changed to:
carbon.conf file: https://gist.github.com/0a4ad596e58899260c50

relay-rules.conf: https://gist.github.com/d26a80bd2f5baf3a6a48

When I try to run carbon-relay.py:

Starting carbon-relay (instance a)
Traceback (most recent call last):
  File "/opt/graphite/bin/carbon-relay.py", line 30, in <module>
    run_twistd_plugin(__file__)
  File "/opt/graphite/lib/carbon/util.py", line 81, in run_twistd_plugin
    runApp(config)
  File "/usr/lib64/python2.6/site-packages/twisted/scripts/twistd.py", line 23, in runApp
    _SomeApplicationRunner(config).run()
  File "/usr/lib64/python2.6/site-packages/twisted/application/app.py", line 411, in run
    self.application = self.createOrGetApplication()
  File "/usr/lib64/python2.6/site-packages/twisted/application/app.py", line 489, in createOrGetApplication
    ser = plg.makeService(self.config.subOptions)
  File "/opt/graphite/lib/twisted/plugins/carbon_relay_plugin.py", line 21, in makeService
    return service.createRelayService(options)
  File "/opt/graphite/lib/carbon/service.py", line 177, in createRelayService
    router = RelayRulesRouter(settings["relay-rules"])
  File "/opt/graphite/lib/carbon/routers.py", line 24, in __init__
    self.rules = loadRelayRules()
TypeError: loadRelayRules() takes exactly 1 argument (0 given)

This happens even if I specify the rules file with --rules

If I look at routers.py here is the offending line:

    self.rules = loadRelayRules()

So I hard code in the path to my rules files and try to start carbon-
relay:

Starting carbon-relay (instance a)
Traceback (most recent call last):
  File "/opt/graphite/bin/carbon-relay.py", line 30, in <module>
    run_twistd_plugin(__file__)
  File "/opt/graphite/lib/carbon/util.py", line 81, in run_twistd_plugin
    runApp(config)
  File "/usr/lib64/python2.6/site-packages/twisted/scripts/twistd.py", line 23, in runApp
    _SomeApplicationRunner(config).run()
  File "/usr/lib64/python2.6/site-packages/twisted/application/app.py", line 411, in run
    self.application = self.createOrGetApplication()
  File "/usr/lib64/python2.6/site-packages/twisted/application/app.py", line 489, in createOrGetApplication
    ser = plg.makeService(self.config.subOptions)
  File "/opt/graphite/lib/twisted/plugins/carbon_relay_plugin.py", line 21, in makeService
    return service.createRelayService(options)
  File "/opt/graphite/lib/carbon/service.py", line 177, in createRelayService
    router = RelayRulesRouter(settings["relay-rules"])
  File "/opt/graphite/lib/carbon/routers.py", line 24, in __init__
    self.rules = loadRelayRules(self.rules_path)
  File "/opt/graphite/lib/carbon/relayrules.py", line 26, in loadRelayRules
    "'destinations' list" % (path, section))
ValueError: Rules file /opt/graphite/conf/relay-rules.conf section np does not define a 'destinations' list

Ok, I did not know destinations was supposed to be in the relay-
rules.conf, I thought that was supposed to be in carbon.conf. Anyway,
just to see what happens, I add destinations to the relay-rules.conf
file so it looks like this: https://gist.github.com/9ccff41a947f693d699d

I also tried this: https://gist.github.com/c0a443af31ce0cedf38a

With both of those in place, carbon-relay finally starts:
Starting carbon-relay (instance a)

here are the logs:
console.log for carbon-relay-a:
03/10/2011 21:46:52 :: Log opened.
03/10/2011 21:46:52 :: twistd 8.2.0 (/usr/bin/python 2.6.5) starting up.
03/10/2011 21:46:52 :: reactor class: twisted.internet.epollreactor.EPollReactor.
03/10/2011 21:46:52 :: twisted.internet.protocol.ServerFactory starting on 2003
03/10/2011 21:46:52 :: Starting factory <twisted.internet.protocol.ServerFactory instance at 0x10ed7a0>
03/10/2011 21:46:52 :: twisted.internet.protocol.ServerFactory starting on 2004
03/10/2011 21:46:52 :: Starting factory <twisted.internet.protocol.ServerFactory instance at 0x112a0e0>
03/10/2011 21:46:52 :: Starting factory CarbonClientFactory(127.0.0.1:2007:prod)
03/10/2011 21:46:52 :: Starting factory CarbonClientFactory(127.0.0.1:2005:np)
03/10/2011 21:47:52 :: Unhandled error in Deferred:
03/10/2011 21:47:52 :: Unhandled Error
Traceback (most recent call last):
  File "/usr/lib64/python2.6/site-packages/twisted/internet/base.py", line 1128, in run
    self.mainLoop()
  File "/usr/lib64/python2.6/site-packages/twisted/internet/base.py", line 1137, in mainLoop
    self.runUntilCurrent()
  File "/usr/lib64/python2.6/site-packages/twisted/internet/base.py", line 757, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "/usr/lib64/python2.6/site-packages/twisted/internet/task.py", line 114, in __call__
    d = defer.maybeDeferred(self.f, *self.a, **self.kw)
--- <exception caught here> ---
  File "/usr/lib64/python2.6/site-packages/twisted/internet/defer.py", line 106, in maybeDeferred
    result = f(*args, **kw)
  File "/opt/graphite/lib/carbon/instrumentation.py", line 100, in recordMetrics
    prefix = 'destinations.%s.' % receiverProto.destinationName
exceptions.AttributeError: MetricLineReceiver instance has no attribute 'destinationName'

carbon-relay is accepting metrics, the individual carbon-cache processes
are running and configured correctly and I think I've followed
everything from the documentation, so I have no idea what I am doing
wrong. We got to about 400,000 metrics/minute off of a single carbon-
cache instance before stuff starting hitting the fan so it seems like
we'll need to roll out a few more servers and instances to keep up with
the data input.

Any insight or advice would be much appreciated.

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.