← Back to team overview

cf-charmers team mailing list archive

Re: Log aggregation with ELK bundle.

 

Thanks for this. I look forward to trying it out. I'll try to post you a
review by EOD tomorrow, once I've had a chance to look at it in more depth
I can comment.

Best,
Ben

On Mon, Nov 10, 2014 at 4:33 PM, Alexander Lomov <lomov.as@xxxxxxxxx> wrote:

> Hey, all.
>
> Today I've deployed ELK and CF charms and got logs from CF components.
>
> The solution is that all components store there logs with following path
> `/var/vcap/sys/log/<job-name>.log` [1] and logstash-agent can be easily
> configured to read this files as an input.
>
> logstash-agent charm has config option to add input source from custom
> file [2]. This config file can be set by using `juju set ...` command as it
> is written in README.md [3]. Unfortunately `juju set ...` didn't work for
> me and I needed to change input config file manually (see script [4]). Also
> I needed to add option to parse inputs as json (with "format => json").
>
> Here is a short script that shows how I linked cc component from CF and
> ELK:
> https://gist.github.com/allomov/5fac04df179f64df51d3
>
> Btw logstash-agent charm is still using 1.1.4 version of logstash, there
> was some inconveniences with it because of old docs and config format. I
> believe that Sergey Matykevich was going to create a MP for updating it.
>
> Potentially there is a way to gather all logs using syslog_aggregator CF
> component that will be deployed through cloudfoundry charm. This component
> collects all logs from components and stores them as files [1]. As for me,
> solution with ELK has more benefits. What do you think about it ?
>
>
>
> [1] http://docs.cloudfoundry.org/running/managing-cf/logging.html
> [2]
> http://bazaar.launchpad.net/~lazypower/charms/trusty/logstash-agent/trunk/view/head:/config.yaml#L8
> [3]
> http://bazaar.launchpad.net/~lazypower/charms/trusty/logstash-agent/trunk/view/head:/README.md#L65
> [4]
> https://gist.github.com/allomov/5fac04df179f64df51d3#file-bind-elk-to-cf-sh-L9
>
>
>
> --
> Mailing list: https://launchpad.net/~cf-charmers
> Post to     : cf-charmers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~cf-charmers
> More help   : https://help.launchpad.net/ListHelp
>
>

Follow ups

References