cf-charmers team mailing list archive

[Kapil Thangavelu <kapil.thangavelu@xxxxxxxxxxxxx>]

for existing charms you don't need one the syslog rel... you do
rsyslog-forwarder as a subordinate charm.


On Fri, Sep 5, 2014 at 2:17 PM, Alexander Lomov <lomov.as@xxxxxxxxx> wrote:

> Agree with you, Cory. Thank you for investigating this question.
>
> I've mentioned that there is no appropriate syslog relation in existing
> charms (for rsyslog and logstash). That's why I was planning to update this
> charms. By the way, do you think it would be useful to make them work with
> latest version of charm helpers (I mean services and contexts helpers)?
>
> Also I think that it would be a good idea to have one common relation to
> connect log aggregators with log emitters. In order to do it I will need to
> add new relation called syslog-drain to loggregator_trafficcontroller.
>
> The fact that loggregator_trafficcontroller relations exports data for
> syslog protocol is very useful (I thought it works through protobuf).
>
> Best wishes,
> Alex L.
>
>
>
> On 5 September 2014 20:46, Cory Johns <cory.johns@xxxxxxxxxxxxx> wrote:
>
>> I was taking a closer look at the rsyslog and logstash charms to get a
>> better idea of how the CF log collector charm will work, and I wanted
>> to share some notes.  I also added Charles to this thread because he
>> has done some work on the rsyslog charm, and I thought he could make
>> sure my assumptions about how it works are valid.
>>
>> First, loggregator_trafficcontrol listens on
>> "loggregator_trafficcontroller" interface, using the "host" and
>> "endpoint_port" fields to provide connection information, and then
>> emits logs using the syslog protocol defined in RFC 5424 to whatever
>> connects to it.
>>
>> Second, the rsyslog seems to listen on the "syslog" interface, but
>> doesn't provide anything on that interface other than the implicit
>> "private-address" field and just assumes the standard port of 514 on
>> both TCP and UDP.  The default protocol appears to be RFC 5424.
>> [Charles: Is all of this correct?]
>>
>> Third, the logstash-indexer charm doesn't appear to support the syslog
>> (RFC 5424) procotol, but it should be fairly easy to add it to the
>> charm using the input config given at [1], below.  The config block is
>> a little more complex because it apparently does translation between
>> RFC 3164 and RFC 5424, but it seems like it should just be copy &
>> pasting that into the logstash-indexer charm.  Regarding the
>> logstash-agent charm, I'm certain we don't need it at all.
>>
>> Fourth, we don't want to have to push a CF app to do the log drain, so
>> we'll want an equivalent service in the CF log collector charm that
>> connects to the "host" and "endpoint_port" fields from the
>> "loggregator_trafficcontroller" relation to do the draining.  This
>> part I'm less clear about, but it seems like it should just be a basic
>> transparent proxy?  It will have to be a "male-to-male" proxy, though,
>> because both the LTC and the other end (rsyslog or logstash-indexer)
>> both expect to be connected to, even though the LTC side then emits to
>> whatever connects to it while the other end consumes.  So, the proxy
>> has to do the connecting on both sides.
>>
>>
>> [1]:
>> http://scottfrederick.cfapps.io/blog/2014/02/20/cloud-foundry-and-logstash
>>
>
>
> --
> Mailing list: https://launchpad.net/~cf-charmers
> Post to     : cf-charmers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~cf-charmers
> More help   : https://help.launchpad.net/ListHelp
>
>