cf-charmers team mailing list archive

[Alexander Lomov <lomov.as@xxxxxxxxx>]

Agree with you, Cory. Thank you for investigating this question.

I've mentioned that there is no appropriate syslog relation in existing
charms (for rsyslog and logstash). That's why I was planning to update this
charms. By the way, do you think it would be useful to make them work with
latest version of charm helpers (I mean services and contexts helpers)?

Also I think that it would be a good idea to have one common relation to
connect log aggregators with log emitters. In order to do it I will need to
add new relation called syslog-drain to loggregator_trafficcontroller.

The fact that loggregator_trafficcontroller relations exports data for
syslog protocol is very useful (I thought it works through protobuf).

Best wishes,
Alex L.



On 5 September 2014 20:46, Cory Johns <cory.johns@xxxxxxxxxxxxx> wrote:

> I was taking a closer look at the rsyslog and logstash charms to get a
> better idea of how the CF log collector charm will work, and I wanted
> to share some notes.  I also added Charles to this thread because he
> has done some work on the rsyslog charm, and I thought he could make
> sure my assumptions about how it works are valid.
>
> First, loggregator_trafficcontrol listens on
> "loggregator_trafficcontroller" interface, using the "host" and
> "endpoint_port" fields to provide connection information, and then
> emits logs using the syslog protocol defined in RFC 5424 to whatever
> connects to it.
>
> Second, the rsyslog seems to listen on the "syslog" interface, but
> doesn't provide anything on that interface other than the implicit
> "private-address" field and just assumes the standard port of 514 on
> both TCP and UDP.  The default protocol appears to be RFC 5424.
> [Charles: Is all of this correct?]
>
> Third, the logstash-indexer charm doesn't appear to support the syslog
> (RFC 5424) procotol, but it should be fairly easy to add it to the
> charm using the input config given at [1], below.  The config block is
> a little more complex because it apparently does translation between
> RFC 3164 and RFC 5424, but it seems like it should just be copy &
> pasting that into the logstash-indexer charm.  Regarding the
> logstash-agent charm, I'm certain we don't need it at all.
>
> Fourth, we don't want to have to push a CF app to do the log drain, so
> we'll want an equivalent service in the CF log collector charm that
> connects to the "host" and "endpoint_port" fields from the
> "loggregator_trafficcontroller" relation to do the draining.  This
> part I'm less clear about, but it seems like it should just be a basic
> transparent proxy?  It will have to be a "male-to-male" proxy, though,
> because both the LTC and the other end (rsyslog or logstash-indexer)
> both expect to be connected to, even though the LTC side then emits to
> whatever connects to it while the other end consumes.  So, the proxy
> has to do the connecting on both sides.
>
>
> [1]:
> http://scottfrederick.cfapps.io/blog/2014/02/20/cloud-foundry-and-logstash
>