cf-charmers team mailing list archive
-
cf-charmers team
-
Mailing list archive
-
Message #00607
Re: Log aggregation with ELK bundle.
Hi Ben,
We are almost with no tasks ahead and this item to validate with you guys. Also, some questions from Alex.
Can we do a quick Demo + follow up Hangout on Wednesday 9AM?
Thank you.
> On Nov 10, 2014, at 9:37 PM, Benjamin Saller <benjamin.saller@xxxxxxxxxxxxx> wrote:
>
> Thanks for this. I look forward to trying it out. I'll try to post you a review by EOD tomorrow, once I've had a chance to look at it in more depth I can comment.
>
> Best,
> Ben
>
> On Mon, Nov 10, 2014 at 4:33 PM, Alexander Lomov <lomov.as@xxxxxxxxx <mailto:lomov.as@xxxxxxxxx>> wrote:
> Hey, all.
>
> Today I've deployed ELK and CF charms and got logs from CF components.
>
> The solution is that all components store there logs with following path `/var/vcap/sys/log/<job-name>.log` [1] and logstash-agent can be easily configured to read this files as an input.
>
> logstash-agent charm has config option to add input source from custom file [2]. This config file can be set by using `juju set ...` command as it is written in README.md [3]. Unfortunately `juju set ...` didn't work for me and I needed to change input config file manually (see script [4]). Also I needed to add option to parse inputs as json (with "format => json").
>
> Here is a short script that shows how I linked cc component from CF and ELK:
> https://gist.github.com/allomov/5fac04df179f64df51d3 <https://gist.github.com/allomov/5fac04df179f64df51d3>
>
> Btw logstash-agent charm is still using 1.1.4 version of logstash, there was some inconveniences with it because of old docs and config format. I believe that Sergey Matykevich was going to create a MP for updating it.
>
> Potentially there is a way to gather all logs using syslog_aggregator CF component that will be deployed through cloudfoundry charm. This component collects all logs from components and stores them as files [1]. As for me, solution with ELK has more benefits. What do you think about it ?
>
>
>
> [1] http://docs.cloudfoundry.org/running/managing-cf/logging.html <http://docs.cloudfoundry.org/running/managing-cf/logging.html>
> [2] http://bazaar.launchpad.net/~lazypower/charms/trusty/logstash-agent/trunk/view/head:/config.yaml#L8 <http://bazaar.launchpad.net/~lazypower/charms/trusty/logstash-agent/trunk/view/head:/config.yaml#L8>
> [3] http://bazaar.launchpad.net/~lazypower/charms/trusty/logstash-agent/trunk/view/head:/README.md#L65 <http://bazaar.launchpad.net/~lazypower/charms/trusty/logstash-agent/trunk/view/head:/README.md#L65>
> [4] https://gist.github.com/allomov/5fac04df179f64df51d3#file-bind-elk-to-cf-sh-L9 <https://gist.github.com/allomov/5fac04df179f64df51d3#file-bind-elk-to-cf-sh-L9>
>
>
>
> --
> Mailing list: https://launchpad.net/~cf-charmers <https://launchpad.net/~cf-charmers>
> Post to : cf-charmers@xxxxxxxxxxxxxxxxxxx <mailto:cf-charmers@xxxxxxxxxxxxxxxxxxx>
> Unsubscribe : https://launchpad.net/~cf-charmers <https://launchpad.net/~cf-charmers>
> More help : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>
>
>
> --
> Mailing list: https://launchpad.net/~cf-charmers
> Post to : cf-charmers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~cf-charmers
> More help : https://help.launchpad.net/ListHelp
Follow ups
References