graphite-dev team mailing list archive

[Chris Davis <chrismd@xxxxxxxxx>]

We seem to be going down very similar paths. I was actually thinking of
having all of Graphite's APIs (rendering graphs, fetching data, searching
the hierarchy, etc) accessible via AMQP, maybe using something like Thrift.
It would also be a great way to add administrative controls to carbon
(graceful shutdown, start/stop listeners, clear cache, reload config,
etc...). But thats a ways down the road still.

-Chris

On Fri, Feb 12, 2010 at 4:11 PM, Brinley, Chris <Chris.Brinley@xxxxxxxxxx>wrote:

>  Internally working towards replacing the links between the persistence
> layer and the routing layer with AMQP. The idea here would be you have
> certain sets of servers that fill a logical storage role such as "dev
> metrics" or "bussiness metrics" should be tied to a message queue of the
> same "topic". Adding and removing capacity in each logical domain then
> becomes a matter of connecting to the queue. Also be between storage layer
> and presentation layer I am working on an implementatation along the same
> lines:
>
> presenation layer requests data about metrics X,Y,Z from the data proivder
> queue. There are some concerns I have with managing response times here but
> that's the 20K foot view.
>
> this also starts to make graphite more serivce oriented in that arbitrary
> app can consume data about metrics.
>
> Chris your familiar with the infrastructure, but i agree sending directly
> to the storage layer via AMQP would be optimal. thats probably not going to
> happen in our case so i think adding it in between the routing and
> storage layers now and side steping carbon entirely in the future may a more
> practical path for us.
>
>  Chris Brinley
>   ------------------------------
> *From:* graphite-dev-bounces+chris.brinley=orbitz.com@xxxxxxxxxxxxxxxxxxx[graphite-dev-bounces+chris.brinley=
> orbitz.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of Chris Davis [
> chrismd@xxxxxxxxx]
> *Sent:* Friday, February 12, 2010 4:00 PM
> *To:* graphite-dev
> *Subject:* [Graphite-dev] request for comments - using amqp and replacing
> carbon-relay
>
>  Hey everyone, recently we have gotten some really great community
> contributions, one of which adds support for the AMQP messaging protocol to
> carbon. I have started migrating some of my systems to using this and I
> think it is a great fit for graphite. If you aren't familiar with it already
> I highly recommend reading
> http://blogs.digitar.com/jjww/2009/01/rabbits-and-warrens/ for a brief
> introduction.
>
>  One area I think AMQP can be especially useful is with a clustered
> graphite installation. Most of you are probably not familiar with Graphite's
> clustering capabilities because I have not documented it at all (sorry, hope
> to change that soon). But essentially, you just setup N different
> independent graphite servers and then configure the webapps to know about
> one another. They then share data in a manner transparent to the user,
> making each server look like one big graphite installation instead of
> several independent ones. The tricky part is partitioning your metrics
> across the servers. Thus far I've solved this problem with a daemon called
> carbon-relay.py, which basically acts as an application-level load balancer
> for your metrics. You are supposed to send all of your data to carbon-relay,
> who then looks up some rules you've given it to decide which carbon-cache(s)
> to relay each metric on to.
>
>  With AMQP, there seems to be a much simpler way to solve this problem.
> Topic exchanges use a dot-delimited naming structure that supports pattern
> matching very similar to graphite's. Basically, you could just publish
> messages containing a value and a timestamp and use the metric name as the
> routing key. Then each carbon-cache can consume from the exchange with one
> or more binding patterns. For the simplest case of having only one server
> the binding pattern would simply be "#" (which is the same as using a fanout
> exchange). For more complex cases you could control what data goes to what
> server by means of configuring each carbon-cache's binding patterns. This
> would effectively replace carbon-relay, and I believe, solve the problem in
> a more robust way.
>
>  This is a bit different than they way it currently works in trunk so I
> wanted to run it by everyone and see what your thoughts are, especially if
> you are already using AMQP or carbon-relay. I am currently in the process of
> testing this configuration and if it works well I will try it in my
> production system. If that goes well, I would like to include the new
> behavior in this month's release. So please send me any comments, questions,
> concerns, etc.
>
>  If you don't plan on using AMQP, that's fine too, the old interface is
> not going away.
>
>  -Chris
>