← Back to team overview

cf-charmers team mailing list archive

Re: Logstash and cloudoundry charms integration status

 

Hi Ben, 

Could we Hangout tomorrow 9 AM PT? It will be much faster than these threads.


On Nov 4, 2014, at 1:46 PM, Benjamin Saller <benjamin.saller@xxxxxxxxxxxxx> wrote:

> Hi,
> 
> That link directed me to your internal Outlook and I was unable to read the thread. 
Yeap, wrong link. -> https://groups.google.com/a/cloudfoundry.org/forum/#!topic/vcap-dev/FE_w5xDG-dg



> 
> Thanks for digging into this. We have two goals here which might help to think about in deciding what to do.
> 
> 1. We want an easy way to see what CF is doing in terms of logging by passing those logs to a analysis tool.
> 
> 2. We want a path to recommend to users that they can connect their applications loggregator streams to a bundle (maybe even the same bundle as in #1) that can help with application specific logging in a Juju world. 
> 
> 
> #2 Should be possible today and examples of how to do that and structure the connection between a deployed app and the log analysis tool would be welcome for early users of this system.
> 
> If Loggregator can't currently fulfil #1 we can investigate that later. Firehosse sounds a little scary as it includes both the CF internal logs and the DEA level logs which we wouldn't normally want to combine and share outside of CF admins. Still a properly configured log filtering package could deal with that.
> 
> So, my take is that for now if we can show a compelling story around Juju bundles and case #2, application specific logging, we should do that now. When we add support for 192+ (and there is work ongoing to help resolve our issues with bosh packages exports that have prevented us from tracking releases) we can revisit #1.


> 
> Thanks, 
> Ben 
> 
> On Tue, Nov 4, 2014 at 8:27 AM, Sergey Matyukevich <s.matyukevich@xxxxxxxxx> wrote:
> Hello all.
> 
> Recently, I've been working for a while on integrating logstash with cloudfoundry charms. I dig into loggregator source code, and the only way that I find out how to obtain all application logs, was to listen to "firehose" websocket endpoint. The problem here is that accordingly to this thread https://groups.google.com/a/cloudfoundry.org/forum/#!topic/vcap-dev/FE_w5xDG-dg "firehose" feature would be only added in 192 cf release. So now I don't know how to continue work on this task, I see the following options:
> 
> 1. Maybe I miss something in loggregator sources and there are some other ways how to obtain all application logs. If you know about such ways, please guide me in the right direction.
> 2. We can wait until this feature would be released, and then update cloudfounry charm to use 192 cf release.
> 3. We can try to update cloudfoundry charm to use develop branch of cf release.
> 4. Maybe you will have some other suggestions.
> 
> --
> Mailing list: https://launchpad.net/~cf-charmers
> Post to     : cf-charmers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~cf-charmers
> More help   : https://help.launchpad.net/ListHelp
> 
> 
> -- 
> Mailing list: https://launchpad.net/~cf-charmers
> Post to     : cf-charmers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~cf-charmers
> More help   : https://help.launchpad.net/ListHelp


Follow ups

References