fuel-dev team mailing list archive

[Dmitry Nikishov <nikishov.da@xxxxxxxxx>]

Mike,

not really, we had a talk with Nikolay Markov and he's actually fine with
leaving this code as it is for 5.0.


2014-04-14 18:08 GMT+04:00 Mike Scherbakov <mscherbakov@xxxxxxxxxxxx>:

> Vladimir,
> is anyone helping out with review request to fuel-library?
>
> Dmitry, I see -1 in this fuel-web part:
> https://review.openstack.org/#/c/85645/2. Are you working on this still?
>
>
> On Mon, Apr 14, 2014 at 5:48 PM, Dmitry Nikishov <nikishov.da@xxxxxxxxx>wrote:
>
>> Guys,
>>
>> I'd like to remind you about the feature freeze for Monitoring system.
>> Could you please review and/or merge it's support into fuel-library?
>>
>>
>> 2014-04-10 21:44 GMT+04:00 Andrew Woodward <xarses@xxxxxxxxx>:
>>
>> Dmitry,
>>>
>>> For future reference, it would be easier to find all of you open commits
>>> for this if you tracked them with a topic. This is most simply done by
>>> submitting the review request with the changes in a git branch other than
>>> master. We commonly use bug/<bug number> or bp/<blueprint-name>. This way
>>> we can search for them in gerrit.
>>>
>>>
>>>
>>> On Mon, Apr 7, 2014 at 12:55 AM, Dmitry Nikishov <nikishov.da@xxxxxxxxx>wrote:
>>>
>>>> I uploaded a change request to fuel-web containing automatic nodes
>>>> removal from zabbix in nailgun: https://review.openstack.org/#/c/85645/
>>>>
>>>> There's also a change request for fuel-library: zabbix class
>>>> declaration. https://review.openstack.org/#/c/85657/
>>>>
>>>> Zabbix 2.2 packages are on their way, launchpad bug:
>>>> https://bugs.launchpad.net/fuel/+bug/1297163
>>>>
>>>> Please review all the other zabbix-related commits because code freeze
>>>> date is not very far. The full list can be found on the blueprint's page:
>>>> https://blueprints.launchpad.net/fuel/+spec/monitoring-system
>>>>
>>>>
>>>> 2014-04-02 13:20 GMT+04:00 Dmitry Nikishov <nikishov.da@xxxxxxxxx>:
>>>>
>>>> Minor improvements (like no more class requiring other classes) have
>>>>> been introduced to fuel-library change requests and changes rebased. All
>>>>> the commits should be up to date now (no more [OUTDATED]).
>>>>>
>>>>> I've also uploaded a request to fuel-web:
>>>>> https://review.openstack.org/#/c/84408/
>>>>> It adds zabbix-server role and necessary fields to deployment facts.
>>>>> It also introduces changes to deployment serializers so zabbix-server role
>>>>> will be deployed first. There are also UI options for zabbix credentials
>>>>> now.
>>>>>
>>>>> Let me know what you think.
>>>>>
>>>>>
>>>>> 2014-03-24 17:02 GMT+04:00 Vladimir Kuklin <vkuklin@xxxxxxxxxxxx>:
>>>>>
>>>>>> Thank you, Dmitry
>>>>>>
>>>>>> We'll look into these, but do not forget to poke us :-)
>>>>>>
>>>>>>
>>>>>> On Mon, Mar 24, 2014 at 10:33 AM, Dmitry Nikishov <
>>>>>> nikishov.da@xxxxxxxxx> wrote:
>>>>>>
>>>>>>> I have split this reimplementation into multiple commits:
>>>>>>>
>>>>>>> https://review.openstack.org/79566    Zabbix server installation
>>>>>>> https://review.openstack.org/81217    Add custom types for zabbix
>>>>>>> configuration Add basic server config
>>>>>>> https://review.openstack.org/81723    Zabbix agent installation
>>>>>>> Basic OS monitoring
>>>>>>> https://review.openstack.org/81754    Add nova monitoring with
>>>>>>> zabbix
>>>>>>> https://review.openstack.org/81765    Keystone monitoring with
>>>>>>> zabbix Glance monitoring with zabbix
>>>>>>> https://review.openstack.org/82036    cinder and swift monitoring
>>>>>>> with zabbix
>>>>>>> https://review.openstack.org/82049    memcached, mysql, horizon and
>>>>>>> rabbit monitoring with zabbix
>>>>>>> https://review.openstack.org/82067    misc services monitoring with
>>>>>>> zabbix
>>>>>>> https://review.openstack.org/82433    Neutron monitoring with zabbix
>>>>>>>
>>>>>>> Would like to get feedback on these. Please note that huge line
>>>>>>> count is caused mainly by xml templates which are used to configure
>>>>>>> monitoring.
>>>>>>>
>>>>>>>
>>>>>>> 2014-03-11 13:29 GMT+04:00 Dmitry Nikishov <nikishov.da@xxxxxxxxx>:
>>>>>>>
>>>>>>> Alright, there's the first part of it: server installation.
>>>>>>>> https://review.openstack.org/#/c/79566/
>>>>>>>>
>>>>>>>> Would appreciate any feedback on it.
>>>>>>>>
>>>>>>>>
>>>>>>>> 2014-03-07 14:40 GMT+04:00 Vladimir Kuklin <vkuklin@xxxxxxxxxxxx>:
>>>>>>>>
>>>>>>>> Dmitry
>>>>>>>>>
>>>>>>>>> It would be awesome if you could split the request into several
>>>>>>>>> ones. E. g. one for server installation. One for each openstack service
>>>>>>>>> template. One for adding templates to the hosts. Thus, it will allow us to
>>>>>>>>> merge it smoothly without a headache.
>>>>>>>>> 07 марта 2014 г. 14:21 пользователь "Dmitry Nikishov" <
>>>>>>>>> nikishov.da@xxxxxxxxx> написал:
>>>>>>>>>
>>>>>>>>>  There's a basic version of my own reimplementation available for
>>>>>>>>>> review:
>>>>>>>>>> https://review.openstack.org/#/c/78919/
>>>>>>>>>> Configuration templates were taken from PL team's module with
>>>>>>>>>> their permission.
>>>>>>>>>> It's able to deploy zabbix server and configure zabbix agents to
>>>>>>>>>> monitor openstack services. It still requires lots of work so it should not
>>>>>>>>>> be merged, but it would be great to get some feedback on it.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2014-02-21 14:55 GMT+04:00 Roman Zhnichkov <
>>>>>>>>>> rzhnichkov@xxxxxxxxxxxx>:
>>>>>>>>>>
>>>>>>>>>>> Andrew,
>>>>>>>>>>>   thanks for this catch, we'll investigate alternatives.
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Roman Zhnichkov
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Feb 20, 2014 at 11:00 PM, Andrew Woodward <
>>>>>>>>>>> xarses@xxxxxxxxx> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I've reviewed most of the zabbix commit request and I see that
>>>>>>>>>>>> it
>>>>>>>>>>>> contains many uses of stored configs which wont work any longer
>>>>>>>>>>>> since
>>>>>>>>>>>> we don't have the puppet db / master anymore so there is likely
>>>>>>>>>>>> a bit
>>>>>>>>>>>> of work there. Additionally we should not merge it because it's
>>>>>>>>>>>> licensed under AGPL which at this point will not be accepted.
>>>>>>>>>>>> I've
>>>>>>>>>>>> reached out to the upstream authors and have requested a change
>>>>>>>>>>>> in
>>>>>>>>>>>> license but this should not be expected. We should be reviewing
>>>>>>>>>>>> additional module providers that have a more suitable upstream
>>>>>>>>>>>> license
>>>>>>>>>>>> and possibly less work to work in our environment.
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Feb 19, 2014 at 11:34 AM, Dmitry Borodaenko
>>>>>>>>>>>> <dborodaenko@xxxxxxxxxxxx> wrote:
>>>>>>>>>>>> > David,
>>>>>>>>>>>> >
>>>>>>>>>>>> > As far as I understand the difference between #4 and #1 is
>>>>>>>>>>>> that in the
>>>>>>>>>>>> > case of #4 Zabbix server is not deployed by Fuel (pre-existing
>>>>>>>>>>>> > monitoring server), so our job is to collect the server
>>>>>>>>>>>> details from
>>>>>>>>>>>> > the operator and configure Zabbix agents to report to that
>>>>>>>>>>>> server.
>>>>>>>>>>>> >
>>>>>>>>>>>> > -DmitryB
>>>>>>>>>>>> >
>>>>>>>>>>>> > On Wed, Feb 19, 2014 at 6:56 AM, David Easter <
>>>>>>>>>>>> deaster@xxxxxxxxxxxx> wrote:
>>>>>>>>>>>> >> Use cases 1 & 2 are fine for MVP in the first release, yes.
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Would the mechanism for deploying use case #4 be the same as
>>>>>>>>>>>> use case #1 –
>>>>>>>>>>>> >> just configuring the server to monitor agents in multiple
>>>>>>>>>>>> clouds?
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Thanks,
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> - David J. Easter
>>>>>>>>>>>> >>   Product Line Manager,  Mirantis
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> From: Roman Zhnichkov <rzhnichkov@xxxxxxxxxxxx>
>>>>>>>>>>>> >> Date: Wednesday, February 19, 2014 at 4:30 AM
>>>>>>>>>>>> >> To: David Easter <deaster@xxxxxxxxxxxx>
>>>>>>>>>>>> >> Cc: "fuel-dev@xxxxxxxxxxxxxxxxxxx" <
>>>>>>>>>>>> fuel-dev@xxxxxxxxxxxxxxxxxxx>
>>>>>>>>>>>> >> Subject: Re: [Fuel-dev] Zabbix feature review request
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Guys,
>>>>>>>>>>>> >>   in other words there are several use cases for Zabbix
>>>>>>>>>>>> server placement:
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Zabbix server as separate role on dedicated hardware
>>>>>>>>>>>> >> Zabbix server role combined with some other role (compute or
>>>>>>>>>>>> storage, since
>>>>>>>>>>>> >> Zabbix uses MySQL, so we don't want to mix OpenStack and
>>>>>>>>>>>> Zabbix data in the
>>>>>>>>>>>> >> same DB)
>>>>>>>>>>>> >> Zabbix server on fuel-master
>>>>>>>>>>>> >> Remote Zabbix server outside of OpenStack environment
>>>>>>>>>>>> >> Combination of any of 4 previous items (for example, we have
>>>>>>>>>>>> Zabbix server
>>>>>>>>>>>> >> on fuel-master monitoring all existing environments and some
>>>>>>>>>>>> other Zabbix
>>>>>>>>>>>> >> servers monitoring particular OpenStack environments)
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Also we have general requirements:
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> HA for Zabbix server
>>>>>>>>>>>> >> Dynamic configuration (automatic node deletion when removing
>>>>>>>>>>>> node from the
>>>>>>>>>>>> >> environment, requires orchestration patching)
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> As MVP I propose to have use cases 1 and 2 implemented
>>>>>>>>>>>> (Zabbix server as
>>>>>>>>>>>> >> separate role ans Zabbix server role combined with compute
>>>>>>>>>>>> or storage) with
>>>>>>>>>>>> >> dynamic configuration requirement.
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Use cases 3-5 as well as HA for Zabbix are subject to further
>>>>>>>>>>>> >> implementation.
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> We'll update the specification according with that.
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Any comments or suggestions?
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> --
>>>>>>>>>>>> >> Roman Zhnichkov
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> On Tue, Feb 18, 2014 at 8:21 PM, David Easter <
>>>>>>>>>>>> deaster@xxxxxxxxxxxx> wrote:
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> On 2/18/14, 4:23 AM, "Bogdan Dobrelya" <
>>>>>>>>>>>> bdobrelia@xxxxxxxxxxxx> wrote:
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> >On 02/18/2014 12:05 PM, Dmitry Nikishov wrote:
>>>>>>>>>>>> >>> >> From this conversation we got following points:
>>>>>>>>>>>> >>> >> 1. It should be possible to install zabbix as a separate
>>>>>>>>>>>> role
>>>>>>>>>>>> >>> >> 2. Combining zabbix role with other roles should be
>>>>>>>>>>>> supported as well
>>>>>>>>>>>> >>> >> 3. There should be support for configuring zabbix agents
>>>>>>>>>>>> to point them
>>>>>>>>>>>> >>> >> to the remote zabbix server
>>>>>>>>>>>> >>> >> 4. Possible zabbix installation on fuel-master?
>>>>>>>>>>>> >>> >> 5. In case of one zabbix server monitoring multiple
>>>>>>>>>>>> environments, nodes
>>>>>>>>>>>> >>> >> should be separated into hostgroups.
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >> Concerns:
>>>>>>>>>>>> >>> >> 1. perfomance when alot of nodes are being monitored by
>>>>>>>>>>>> single server
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> True.  We’d need to publish the maximum number of nodes
>>>>>>>>>>>> that can be
>>>>>>>>>>>> >>> monitored efficiently by one server.  If the number of
>>>>>>>>>>>> monitored nodes
>>>>>>>>>>>> >>> goes above that number, we’d recommend a dedicated server
>>>>>>>>>>>> within the
>>>>>>>>>>>> >>> largest deployed cloud (for example) to split the burden.
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> >> 2. if zabbix agents were configured to connect to the
>>>>>>>>>>>> remote server
>>>>>>>>>>>> >>> >> (outside the environment), once this env gets deleted,
>>>>>>>>>>>> it's nodes will
>>>>>>>>>>>> >>> >> be rebooted into bootstrap. However, they won't be
>>>>>>>>>>>> deleted from zabbix.
>>>>>>>>>>>> >>> >> It's not clear how to remove them from zabbix server.
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> We’d have to build in automation to do that for the user.
>>>>>>>>>>>>  I.e. if a
>>>>>>>>>>>> >>> remote node is deleted, update the corresponding zabbix
>>>>>>>>>>>> server’s info.
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >> Open question: if there will be support for zabbix on
>>>>>>>>>>>> fuel-master,
>>>>>>>>>>>> >>> >> there
>>>>>>>>>>>> >>> >> should be an option either to enable or disable it
>>>>>>>>>>>> somewhere in the
>>>>>>>>>>>> >>> >>fuelmenu
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >> Any thoughts?
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> Configuration of the “local” zabbix server would need to be
>>>>>>>>>>>> an option,
>>>>>>>>>>>> >>> yes.  It’s very possible that you’d have a situation where
>>>>>>>>>>>> the “local”
>>>>>>>>>>>> >>> zabbix server is watching some clouds while a dedicated
>>>>>>>>>>>> zabbix server
>>>>>>>>>>>> >>> within one of the deployed clouds is watching only the
>>>>>>>>>>>> nodes within that
>>>>>>>>>>>> >>> cloud.  I.e. there can be a mixture.  Fuel would need to
>>>>>>>>>>>> keep track of
>>>>>>>>>>>> >>> which nodes were being monitored by which server(s).
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> >Looks like "2 - Can Zabbix be set up in HA (even in
>>>>>>>>>>>> active/passive)?"
>>>>>>>>>>>> >>> >wasn't addressed by either of the points, e.g.:
>>>>>>>>>>>> >>> >- How can we configure HA zabbix in case of:
>>>>>>>>>>>> >>> >* one of the zabbix nodes is a master node?
>>>>>>>>>>>> >>> >* some of the zabbix nodes were externally configured by
>>>>>>>>>>>> the user
>>>>>>>>>>>> >>> > himself?
>>>>>>>>>>>> >>> >* some of the zabbix nodes belong to the different envs
>>>>>>>>>>>> and combined
>>>>>>>>>>>> >>> >with the another roles.
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >> 2014-02-15 1:16 GMT+04:00 Andrew Woodward <
>>>>>>>>>>>> xarses@xxxxxxxxx
>>>>>>>>>>>> >>> >> <mailto:xarses@xxxxxxxxx>>:
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>     Based on the way the data-points appear to be
>>>>>>>>>>>> collected, there is
>>>>>>>>>>>> >>> >>     definitely a point where monitoring the nodes will
>>>>>>>>>>>> become too
>>>>>>>>>>>> >>> >>     intense for a node sharing roles (or even a single
>>>>>>>>>>>> node) (I've
>>>>>>>>>>>> >>> >>     personally overloaded most monitoring systems in
>>>>>>>>>>>> default
>>>>>>>>>>>> >>> >>     configurations with ~200 nodes.) However a single
>>>>>>>>>>>> pane of glass to
>>>>>>>>>>>> >>> >>     view all clusters is extremely important to the
>>>>>>>>>>>> operations teams.
>>>>>>>>>>>> >>> >> It
>>>>>>>>>>>> >>> >>     is therefor necessary for us to be able to specify
>>>>>>>>>>>> an existing
>>>>>>>>>>>> >>> >>     Zabbix server that may be deployed elsewhere, or
>>>>>>>>>>>> even completely
>>>>>>>>>>>> >>> >>     outside the scope of fuel.
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>     So to get down to it. we should be able to (1)
>>>>>>>>>>>> install Zabbix as a
>>>>>>>>>>>> >>> >>     role (2) separately of the Zabbix role configure
>>>>>>>>>>>> Zabbix agents on
>>>>>>>>>>>> >>> >>     nodes inside a cluster to point to a Zabbix node.
>>>>>>>>>>>> (3) if a Zabbix
>>>>>>>>>>>> >>> >>     role is defined in the cluster assume that it should
>>>>>>>>>>>> be used to
>>>>>>>>>>>> >>> >>     configure (2) but allow it to be replaced anyway.
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>     On Fri, Feb 14, 2014 at 11:42 AM, David Easter
>>>>>>>>>>>> >>> >> <deaster@xxxxxxxxxxxx
>>>>>>>>>>>> >>> >>     <mailto:deaster@xxxxxxxxxxxx>> wrote:
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>         Good observation!  For a situation where the
>>>>>>>>>>>> tenant for each
>>>>>>>>>>>> >>> >>         cloud were
>>>>>>>>>>>> >>> >>         the same, that would certainly make sense, yes.
>>>>>>>>>>>>  Perhaps that
>>>>>>>>>>>> >>> >>         should even
>>>>>>>>>>>> >>> >>         be the default if it doesn’t impact the
>>>>>>>>>>>> abilities of the Fuel
>>>>>>>>>>>> >>> >>         Master node.
>>>>>>>>>>>> >>> >>          I.e. install the Zabbix server on the Fuel Node
>>>>>>>>>>>> by default and
>>>>>>>>>>>> >>> >>         the Zabbix
>>>>>>>>>>>> >>> >>         agents pointing back to the Fuel Master Node by
>>>>>>>>>>>> default.  If
>>>>>>>>>>>> >>> >> the
>>>>>>>>>>>> >>> >>         operator
>>>>>>>>>>>> >>> >>         decides that a given environment needs its own
>>>>>>>>>>>> dedicated Zabbix
>>>>>>>>>>>> >>> >>         server,
>>>>>>>>>>>> >>> >>         the operator could add a role on an existing
>>>>>>>>>>>> node, or a
>>>>>>>>>>>> >>> >>         standalone node,
>>>>>>>>>>>> >>> >>         for a Zabbix server and all agents in the
>>>>>>>>>>>> environment would
>>>>>>>>>>>> >>> >> then
>>>>>>>>>>>> >>> >>         point to
>>>>>>>>>>>> >>> >>         that deployed server.  Then the URL for the
>>>>>>>>>>>> deployed Zabbix
>>>>>>>>>>>> >>> >>         server would
>>>>>>>>>>>> >>> >>         properly appear when the environment were opened.
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>         In answer to your question, though; were a
>>>>>>>>>>>> service provider (or
>>>>>>>>>>>> >>> >>         large IT
>>>>>>>>>>>> >>> >>         organization) to create clouds for its
>>>>>>>>>>>> customers, it may make
>>>>>>>>>>>> >>> >>         more sense
>>>>>>>>>>>> >>> >>         to provide a monitoring service that the
>>>>>>>>>>>> customer could draw
>>>>>>>>>>>> >>> >>from
>>>>>>>>>>>> >>> >>         independently of other customers.  Or in
>>>>>>>>>>>> instances where the
>>>>>>>>>>>> >>> >>         clouds are
>>>>>>>>>>>> >>> >>         highly separated from each other in terms of
>>>>>>>>>>>> data content or
>>>>>>>>>>>> >>> >>         internal
>>>>>>>>>>>> >>> >>         departments.  By creating a Zabbix server per
>>>>>>>>>>>> environment, this
>>>>>>>>>>>> >>> >>         would
>>>>>>>>>>>> >>> >>         address issues of ensuring isolation of access
>>>>>>>>>>>> and data.
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>         Thanks,
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>         - David J. Easter
>>>>>>>>>>>> >>> >>           Product Line Manager
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>         On 2/14/14, 11:32 AM, "Dmitry Borodaenko"
>>>>>>>>>>>> >>> >>         <dborodaenko@xxxxxxxxxxxx <mailto:
>>>>>>>>>>>> dborodaenko@xxxxxxxxxxxx>>
>>>>>>>>>>>> >>> >>wrote:
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>         >What about deploying Zabbix on the Fuel node
>>>>>>>>>>>> itself?
>>>>>>>>>>>> >>> >>         >
>>>>>>>>>>>> >>> >>         >In a scenario where a large number of OpenStack
>>>>>>>>>>>> environments
>>>>>>>>>>>> >>> >> is
>>>>>>>>>>>> >>> >>         >deployed from the same Fuel node, I'm
>>>>>>>>>>>> struggling to come up
>>>>>>>>>>>> >>> >>with a
>>>>>>>>>>>> >>> >>         >scenario where operator would prefer to have a
>>>>>>>>>>>> separate
>>>>>>>>>>>> >>> >>monitoring
>>>>>>>>>>>> >>> >>         >server for each environment, instead of
>>>>>>>>>>>> monitoring all
>>>>>>>>>>>> >>> >>environments
>>>>>>>>>>>> >>> >>         >from a single pane of glass.
>>>>>>>>>>>> >>> >>         >
>>>>>>>>>>>> >>> >>         >-DmitryB
>>>>>>>>>>>> >>> >>         >
>>>>>>>>>>>> >>> >>         >
>>>>>>>>>>>> >>> >>         >On Fri, Feb 14, 2014 at 9:08 AM, David Easter
>>>>>>>>>>>> >>> >>         <deaster@xxxxxxxxxxxx <mailto:
>>>>>>>>>>>> deaster@xxxxxxxxxxxx>>
>>>>>>>>>>>> >>> >>         >wrote:
>>>>>>>>>>>> >>> >>         >> Correct.  Just to understand, there’s growing
>>>>>>>>>>>> concern from
>>>>>>>>>>>> >>> >>         our customer
>>>>>>>>>>>> >>> >>         >> base about how many servers they need in an
>>>>>>>>>>>> OpenStack
>>>>>>>>>>>> >>> >>         environment that
>>>>>>>>>>>> >>> >>         >> don’t add to the compute power of the
>>>>>>>>>>>> environment.  We
>>>>>>>>>>>> >>> >>         already require
>>>>>>>>>>>> >>> >>         >>the
>>>>>>>>>>>> >>> >>         >> Fuel Master Node on its own server, so
>>>>>>>>>>>> _requiring_ a Zabbix
>>>>>>>>>>>> >>> >>         node of its
>>>>>>>>>>>> >>> >>         >> own will be more difficult to justify -
>>>>>>>>>>>> especially in small
>>>>>>>>>>>> >>> >>         environments
>>>>>>>>>>>> >>> >>         >> (5-10 servers).
>>>>>>>>>>>> >>> >>         >>
>>>>>>>>>>>> >>> >>         >> While it is completely proper to point out
>>>>>>>>>>>> that individual
>>>>>>>>>>>> >>> >>         services
>>>>>>>>>>>> >>> >>         >>_can_
>>>>>>>>>>>> >>> >>         >> (and perhaps should) be installed on their
>>>>>>>>>>>> own nodes for
>>>>>>>>>>>> >>> >>         performance
>>>>>>>>>>>> >>> >>         >> reasons, the option to combine them with
>>>>>>>>>>>> other roles on the
>>>>>>>>>>>> >>> >>         same server
>>>>>>>>>>>> >>> >>         >> needs to be an option when performance is not
>>>>>>>>>>>> an issue.  In
>>>>>>>>>>>> >>> >>other
>>>>>>>>>>>> >>> >>         >>words, I
>>>>>>>>>>>> >>> >>         >> should be able to have a Zabbix server
>>>>>>>>>>>> running on the same
>>>>>>>>>>>> >>> >>         server as my
>>>>>>>>>>>> >>> >>         >> compute or controller function.  I should
>>>>>>>>>>>> also be able to
>>>>>>>>>>>> >>> >>         install it on
>>>>>>>>>>>> >>> >>         >> its own standalone server if I’m concerned
>>>>>>>>>>>> about
>>>>>>>>>>>> >>> >> performance.
>>>>>>>>>>>> >>> >>         >>
>>>>>>>>>>>> >>> >>         >> Thanks,
>>>>>>>>>>>> >>> >>         >>
>>>>>>>>>>>> >>> >>         >> - David J. Easter
>>>>>>>>>>>> >>> >>         >>   Product Line Manager,  Mirantis
>>>>>>>>>>>> >>> >>         >>
>>>>>>>>>>>> >>> >>         >>
>>>>>>>>>>>> >>> >>         >> On 2/13/14, 7:49 AM, "Matthew Mosesohn"
>>>>>>>>>>>> >>> >>         <mmosesohn@xxxxxxxxxxxx <mailto:
>>>>>>>>>>>> mmosesohn@xxxxxxxxxxxx>> wrote:
>>>>>>>>>>>> >>> >>         >>
>>>>>>>>>>>> >>> >>         >>>Dmitry,
>>>>>>>>>>>> >>> >>         >>>
>>>>>>>>>>>> >>> >>         >>>I think it would be logical to say that it
>>>>>>>>>>>> should be trivial
>>>>>>>>>>>> >>> >>to
>>>>>>>>>>>> >>> >>         >>>reinstall a separate Zabbit node since it
>>>>>>>>>>>> doesn't contain
>>>>>>>>>>>> >>> >> any
>>>>>>>>>>>> >>> >>         data
>>>>>>>>>>>> >>> >>         >>>critical to the production environment. I
>>>>>>>>>>>> think if you can
>>>>>>>>>>>> >>> >>         remove and
>>>>>>>>>>>> >>> >>         >>>add a new zabbix server to a deployed
>>>>>>>>>>>> environment, it would
>>>>>>>>>>>> >>> >>         meet a lot
>>>>>>>>>>>> >>> >>         >>>of our users' needs.
>>>>>>>>>>>> >>> >>         >>>
>>>>>>>>>>>> >>> >>         >>>
>>>>>>>>>>>> >>> >>         >>>On Thu, Feb 13, 2014 at 6:54 PM, Dmitry
>>>>>>>>>>>> Nikishov
>>>>>>>>>>>> >>> >>         <nikishov.da@xxxxxxxxx <mailto:
>>>>>>>>>>>> nikishov.da@xxxxxxxxx>>
>>>>>>>>>>>> >>> >>         >>>wrote:
>>>>>>>>>>>> >>> >>         >>>> Matthew,
>>>>>>>>>>>> >>> >>         >>>> It should be possible to install Zabbix
>>>>>>>>>>>> alongside with
>>>>>>>>>>>> >>> >>         OpenStack
>>>>>>>>>>>> >>> >>         >>>>components
>>>>>>>>>>>> >>> >>         >>>> on a controller, compute etc. However,
>>>>>>>>>>>> Zabbix isn't an
>>>>>>>>>>>> >>> >>         OpenStack
>>>>>>>>>>>> >>> >>         >>>>component,
>>>>>>>>>>>> >>> >>         >>>> it does not directly affect cluster. So
>>>>>>>>>>>> there's no need to
>>>>>>>>>>>> >>> >>         combine it
>>>>>>>>>>>> >>> >>         >>>>with
>>>>>>>>>>>> >>> >>         >>>> other roles or making it high-available. It
>>>>>>>>>>>> makes sense to
>>>>>>>>>>>> >>> >>         install it
>>>>>>>>>>>> >>> >>         >>>>on a
>>>>>>>>>>>> >>> >>         >>>> separate node.
>>>>>>>>>>>> >>> >>         >>>>
>>>>>>>>>>>> >>> >>         >>>> Bogdan,
>>>>>>>>>>>> >>> >>         >>>> I like the idea of combining metering,
>>>>>>>>>>>> monitoring and
>>>>>>>>>>>> >>> >>         logging on one
>>>>>>>>>>>> >>> >>         >>>>node.
>>>>>>>>>>>> >>> >>         >>>> However, I think it would be better to have
>>>>>>>>>>>> separate roles
>>>>>>>>>>>> >>> >>         for all
>>>>>>>>>>>> >>> >>         >>>>three of
>>>>>>>>>>>> >>> >>         >>>> them. But these roles should not conflict
>>>>>>>>>>>> with each other.
>>>>>>>>>>>> >>> >>         This way it
>>>>>>>>>>>> >>> >>         >>>>will
>>>>>>>>>>>> >>> >>         >>>> be more customizable.
>>>>>>>>>>>> >>> >>         >>>>
>>>>>>>>>>>> >>> >>         >>>> 2014-02-13 16:07 GMT+03:00 Bogdan Dobrelya
>>>>>>>>>>>> >>> >>         <bdobrelia@xxxxxxxxxxxx <mailto:
>>>>>>>>>>>> bdobrelia@xxxxxxxxxxxx>>:
>>>>>>>>>>>> >>> >>         >>>>
>>>>>>>>>>>> >>> >>         >>>>> On 02/13/2014 02:33 PM, Matthew Mosesohn
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> >>> >>         >>>>> > Dmitry,
>>>>>>>>>>>> >>> >>         >>>>> >
>>>>>>>>>>>> >>> >>         >>>>> > I have two questions:
>>>>>>>>>>>> >>> >>         >>>>> > 1 - Can Zabbix Server Role be combined
>>>>>>>>>>>> with any other
>>>>>>>>>>>> >>> >>         role? (like
>>>>>>>>>>>> >>> >>         >>>>> > controller)
>>>>>>>>>>>> >>> >>         >>>>> > 2 - Can Zabbix be set up in HA (even in
>>>>>>>>>>>> >>> >> active/passive)?
>>>>>>>>>>>> >>> >>         What are
>>>>>>>>>>>> >>> >>         >>>>>the
>>>>>>>>>>>> >>> >>         >>>>> > obstacles?
>>>>>>>>>>>> >>> >>         >>>>> >
>>>>>>>>>>>> >>> >>         >>>>>
>>>>>>>>>>>> >>> >>         >>>>> And even if it cannot, I believe the good
>>>>>>>>>>>> point is to use
>>>>>>>>>>>> >>> >>         it as
>>>>>>>>>>>> >>> >>         >>>>> Monitoring & Metering & Logging node, see
>>>>>>>>>>>> my comments
>>>>>>>>>>>> >>> >>         inside the doc.
>>>>>>>>>>>> >>> >>         >>>>>
>>>>>>>>>>>> >>> >>         >>>>> > On Thu, Feb 13, 2014 at 4:28 PM, Дмитрий
>>>>>>>>>>>> Никишов
>>>>>>>>>>>> >>> >>         >>>>><nikishov.da@xxxxxxxxx <mailto:
>>>>>>>>>>>> nikishov.da@xxxxxxxxx>>
>>>>>>>>>>>> >>> >>         >>>>> > wrote:
>>>>>>>>>>>> >>> >>         >>>>> >> Hello everyone.
>>>>>>>>>>>> >>> >>         >>>>> >>
>>>>>>>>>>>> >>> >>         >>>>> >> I've been working on Zabbix integration
>>>>>>>>>>>> for a while
>>>>>>>>>>>> >>> >>         now, and I'd
>>>>>>>>>>>> >>> >>         >>>>>like
>>>>>>>>>>>> >>> >>         >>>>> >> know
>>>>>>>>>>>> >>> >>         >>>>> >> what do you guys think about it.
>>>>>>>>>>>> >>> >>         >>>>> >>
>>>>>>>>>>>> >>> >>         >>>>> >> The specification:
>>>>>>>>>>>> >>> >>         >>>>> >>
>>>>>>>>>>>> >>> >>         >>>>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> >>>>>>> >>>>>>>
>>>>>>>>>>>> https://docs.google.com/document/d/1g9xtBrgwgpeNV5YYxTMFu2h-yIK_PC1Z
>>>>>>>>>>>> >>> >>>>>>>JD
>>>>>>>>>>>> >>> >>         >>>>>mw
>>>>>>>>>>>> >>> >>         >>>>>84RquJk
>>>>>>>>>>>> >>> >>         >>>>> >>
>>>>>>>>>>>> >>> >>         >>>>> >> Changes to fuel-library so far:
>>>>>>>>>>>> >>> >>         >>>>>https://review.openstack.org/#/c/73254/
>>>>>>>>>>>> >>> >>         >>>>> >> Changes to fuel-main and fuel-web will
>>>>>>>>>>>> be coming soon.
>>>>>>>>>>>> >>> >>         >>>>> >>
>>>>>>>>>>>> >>> >>         >>>>> >> There is an open question related to UI
>>>>>>>>>>>> - parameter
>>>>>>>>>>>> >>> >>         validation:
>>>>>>>>>>>> >>> >>         >>>>> >> - there should be only one
>>>>>>>>>>>> zabbix-server node
>>>>>>>>>>>> >>> >>         >>>>> >> - "Install Zabbix" checkbox should only
>>>>>>>>>>>> be available
>>>>>>>>>>>> >>> >> if
>>>>>>>>>>>> >>> >>         there is a
>>>>>>>>>>>> >>> >>         >>>>>node
>>>>>>>>>>>> >>> >>         >>>>> >> with
>>>>>>>>>>>> >>> >>         >>>>> >> "zabbix-server" role
>>>>>>>>>>>> >>> >>         >>>>> >>
>>>>>>>>>>>> >>> >>         >>>>> >> --
>>>>>>>>>>>> >>> >>         >>>>> >> Regards
>>>>>>>>>>>> >>> >>         >>>>> >> Dmitry Nikishov
>>>>>>>>>>>> >>> >>         >>>>> >>
>>>>>>>>>>>> >>> >>         >>>>> >> --
>>>>>>>>>>>> >>> >>         >>>>> >> Mailing list:
>>>>>>>>>>>> https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> >>         >>>>> >> Post to     :
>>>>>>>>>>>> fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> >>> >>         <mailto:fuel-dev@xxxxxxxxxxxxxxxxxxx>
>>>>>>>>>>>> >>> >>         >>>>> >> Unsubscribe :
>>>>>>>>>>>> https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> >>         >>>>> >> More help   :
>>>>>>>>>>>> https://help.launchpad.net/ListHelp
>>>>>>>>>>>> >>> >>         >>>>> >>
>>>>>>>>>>>> >>> >>         >>>>> >
>>>>>>>>>>>> >>> >>         >>>>>
>>>>>>>>>>>> >>> >>         >>>>>
>>>>>>>>>>>> >>> >>         >>>>> --
>>>>>>>>>>>> >>> >>         >>>>> Best regards,
>>>>>>>>>>>> >>> >>         >>>>> Bogdan Dobrelya,
>>>>>>>>>>>> >>> >>         >>>>> Skype #bogdando_at_yahoo.com
>>>>>>>>>>>> >>> >><http://bogdando_at_yahoo.com>
>>>>>>>>>>>> >>> >>         >>>>> Irc #bogdando
>>>>>>>>>>>> >>> >>         >>>>
>>>>>>>>>>>> >>> >>         >>>>
>>>>>>>>>>>> >>> >>         >>>>
>>>>>>>>>>>> >>> >>         >>>>
>>>>>>>>>>>> >>> >>         >>>> --
>>>>>>>>>>>> >>> >>         >>>> Regards
>>>>>>>>>>>> >>> >>         >>>> Dmitry Nikishov
>>>>>>>>>>>> >>> >>         >>>
>>>>>>>>>>>> >>> >>         >>>--
>>>>>>>>>>>> >>> >>         >>>Mailing list: https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> >>         >>>Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> >>> >>         <mailto:fuel-dev@xxxxxxxxxxxxxxxxxxx>
>>>>>>>>>>>> >>> >>         >>>Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> >>         >>>More help   :
>>>>>>>>>>>> https://help.launchpad.net/ListHelp
>>>>>>>>>>>> >>> >>         >>
>>>>>>>>>>>> >>> >>         >>
>>>>>>>>>>>> >>> >>         >>
>>>>>>>>>>>> >>> >>         >> --
>>>>>>>>>>>> >>> >>         >> Mailing list: https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> >>         >> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> >>> >>         <mailto:fuel-dev@xxxxxxxxxxxxxxxxxxx>
>>>>>>>>>>>> >>> >>         >> Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> >>         >> More help   :
>>>>>>>>>>>> https://help.launchpad.net/ListHelp
>>>>>>>>>>>> >>> >>         >
>>>>>>>>>>>> >>> >>         >
>>>>>>>>>>>> >>> >>         >
>>>>>>>>>>>> >>> >>         >--
>>>>>>>>>>>> >>> >>         >Dmitry Borodaenko
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>         --
>>>>>>>>>>>> >>> >>         Mailing list: https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> >>         Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> >>> >>         <mailto:fuel-dev@xxxxxxxxxxxxxxxxxxx>
>>>>>>>>>>>> >>> >>         Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> >>         More help   :
>>>>>>>>>>>> https://help.launchpad.net/ListHelp
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>     --
>>>>>>>>>>>> >>> >>     If google has done it, Google did it right!
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>     --
>>>>>>>>>>>> >>> >>     Mailing list: https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> >>     Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> >>> >>     <mailto:fuel-dev@xxxxxxxxxxxxxxxxxxx>
>>>>>>>>>>>> >>> >>     Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> >>     More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >> --
>>>>>>>>>>>> >>> >> Regards
>>>>>>>>>>>> >>> >> Dmitry Nikishov
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >> This body part will be downloaded on demand.
>>>>>>>>>>>> >>> >>
>>>>>>>>>>>> >>> >
>>>>>>>>>>>> >>> >
>>>>>>>>>>>> >>> >--
>>>>>>>>>>>> >>> >Best regards,
>>>>>>>>>>>> >>> >Bogdan Dobrelya,
>>>>>>>>>>>> >>> >Skype #bogdando_at_yahoo.com
>>>>>>>>>>>> >>> >Irc #bogdando
>>>>>>>>>>>> >>> >
>>>>>>>>>>>> >>> >--
>>>>>>>>>>>> >>> >Mailing list: https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> >Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> >>> >Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> >More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> --
>>>>>>>>>>>> >>> Mailing list: https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> >>> Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> --
>>>>>>>>>>>> >> Mailing list: https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> >> Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>>>>>>> >> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>> >>
>>>>>>>>>>>> >
>>>>>>>>>>>> >
>>>>>>>>>>>> >
>>>>>>>>>>>> > --
>>>>>>>>>>>> > Dmitry Borodaenko
>>>>>>>>>>>> >
>>>>>>>>>>>> > --
>>>>>>>>>>>> > Mailing list: https://launchpad.net/~fuel-dev
>>>>>>>>>>>> > Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> > Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>>>>>>> > More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> If google has done it, Google did it right!
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Mailing list: https://launchpad.net/~fuel-dev
>>>>>>>>>>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Mailing list: https://launchpad.net/~fuel-dev
>>>>>>>>>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Regards
>>>>>>>>>> Dmitry Nikishov
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Mailing list: https://launchpad.net/~fuel-dev
>>>>>>>>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Regards
>>>>>>>> Dmitry Nikishov
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Regards
>>>>>>> Dmitry Nikishov
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Yours Faithfully,
>>>>>> Vladimir Kuklin,
>>>>>> Fuel Library Tech Lead,
>>>>>> Mirantis, Inc.
>>>>>> +7 (495) 640-49-04
>>>>>> +7 (926) 702-39-68
>>>>>> Skype kuklinvv
>>>>>> 45bk3, Vorontsovskaya Str.
>>>>>> Moscow, Russia,
>>>>>> www.mirantis.com <http://www.mirantis.ru/>
>>>>>> www.mirantis.ru
>>>>>> vkuklin@xxxxxxxxxxxx
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards
>>>>> Dmitry Nikishov
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Regards
>>>> Dmitry Nikishov
>>>>
>>>> --
>>>> Mailing list: https://launchpad.net/~fuel-dev
>>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>>
>>> --
>>> Andrew
>>> Mirantis
>>> Ceph community
>>>
>>
>>
>>
>> --
>> Regards
>> Dmitry Nikishov
>>
>> --
>> Mailing list: https://launchpad.net/~fuel-dev
>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~fuel-dev
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
> Mike Scherbakov
> #mihgen
>



-- 
Regards
Dmitry Nikishov