cf-charmers team mailing list archive

[Kapil Thangavelu <kapil.thangavelu@xxxxxxxxxxxxx>]

oh.. for rsyslog forwarding, you'll need to setup a syslog input on that
charm, or aggregate to rsyslog-aggregator and then parse files off with an
agent to send to logstash indexer. i'd be careful as direct rsyslog may
backlog logstash indexer depending on volume flow, using redis as a buffer
may help.

-k


On Tue, Oct 21, 2014 at 2:55 PM, Kapil Thangavelu <
kapil.thangavelu@xxxxxxxxxxxxx> wrote:

> the input relation name has an interface redis, ie it connects to redis,
> ie logstash implements the redis server protocol
>
> bundle example
> http://bazaar.launchpad.net/~lazypower/charms/bundles/big-data-logger/bundle/view/head:/bundles.yaml
>
> -k
>
>
>
>
>
> On Tue, Oct 21, 2014 at 2:27 PM, Alexander Lomov <lomov.as@xxxxxxxxx>
> wrote:
>
>> Hey, Kapil.
>>
>> Thank you for pointing out a new versions of logstash charms. I've got
>> versions of a charm that I've used from first lines of a search in charm
>> store, I didn't thought they could be outdated so much.
>>
>> I've looked at new charm today and have some questions. Logstash charm is
>> in python (yey!) and uses logstash v1.4.2, but doesn't have connection to
>> redis (strange thing: it "provides" a relation called "input"
>> https://bazaar.launchpad.net/~lazypower/charms/trusty/logstash/trunk/view/head:/metadata.yaml#L11).
>> logstash-agent still used logstash v1.1.4, but it might be ok.
>>
>> The question now is: do you have any working bundle with to test work of
>> log aggregation charms for logstash and rsyslog? It would help me a lot to
>> understand how they should work together.
>>
>>
>> Thank you 1 more time,
>> Alex L.
>>
>>
>>
>> On 21 October 2014 01:48, Kapil Thangavelu <
>> kapil.thangavelu@xxxxxxxxxxxxx> wrote:
>>
>>> per email update on this earlier today, recent versions of logstash in
>>> dev are at
>>>
>>> cs:~lazypower/trusty/logstash-agent
>>> cs:~lazypower/trusty/logstash
>>>
>>> not sure i would use the agent (jruby + jvm) as its heavyweight, but the
>>> server is reasonable and supports redis inputs per the std scaling semantic
>>> around logstash.
>>>
>>> -k
>>>
>>>
>>> On Mon, Oct 20, 2014 at 6:23 PM, Alexander Lomov <lomov.as@xxxxxxxxx>
>>> wrote:
>>>
>>>> Hey, all.
>>>>
>>>> Today I tried to create juju bundle with
>>>> logstash+elasticsearch+kibana+rails. Here is approximate bundle that I used
>>>> - https://gist.github.com/allomov/088d54229c051a6266b4 (I needed to do
>>>> some stuff manually). Here is my impressions and experience:
>>>>
>>>>    1. logstash charms are outdated:
>>>>       1. current logstash stable version is 1.4.2 vs 1.1/1.2 in the
>>>>       charms;
>>>>       2. charms are written in bash without using charmhelpers;
>>>>       3. embedded elasticsearch of logstash-indexer charm is not
>>>>       compatible with actual version of kibana charm;
>>>>       4. current version of elasticsearch charm doesn't support
>>>>       relation with logstash-indexer.
>>>>    2. logstash-agent is build to use redis embedded
>>>>    in logstash-indexer. I'm not sure that this is correct solution since user
>>>>    may want to connect his redis to charms or create redis cluster. Adding
>>>>    implicit connection of logstash-agent to logstash-indexer through redis
>>>>    will make architecture more clear.
>>>>
>>>> I'm pretty sure that logstash charms need to be updated with latest
>>>> charmhelpers and python implementation. What do you think about it ?
>>>>
>>>> Thank you,
>>>> Alex L.
>>>>
>>>>
>>>> --
>>>> Mailing list: https://launchpad.net/~cf-charmers
>>>> Post to     : cf-charmers@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~cf-charmers
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>
>