fuel-dev team mailing list archive

[Dmitry Borodaenko <dborodaenko@xxxxxxxxxxxx>]

There are two primary problems with running Ceph monitors on controllers:

1) A Ceph bug can cause the ceph-mon process to run away (overload the
CPU or otherwise impact the system), causing crm to fail the whole
controller node with all the other services. When ceph-mon is isolated
to a dedicated host, ceph-mon failures have significantly smaller
chance of disrupting other services in the environment.

2) Scaling ceph-mon cluster should follow storage requirements and
shouldn't be coupled with scaling OpenStack services.

-DmitryB

On Mon, Feb 3, 2014 at 10:03 PM, Roman Alekseenkov
<ralekseenkov@xxxxxxxxxxxx> wrote:
> Based on our experience with Ceph, I would say it should be role-based.
> Meaning, option #2.
>
> We made a decision to always deploy Ceph monitors on controllers (that would
> be an equivalent of option #1) and this decision turned out not to be the
> right one. Dmitry can shed more light on this, I think.
>
> Roman
>
>
> On Mon, Feb 3, 2014 at 9:44 PM, Mike Scherbakov <mscherbakov@xxxxxxxxxxxx>
> wrote:
>>
>> +David, Nadya
>>
>> Hi Max,
>> as we discussed verbally there is a major concern behind - about placement
>> of MongoDB. As I understand right, it is expected that there is a huge disk
>> IO consumption in case of larger deployments (let's say >50 nodes).
>> If it is the case, then I would agree that we may not want to use shared
>> disks for MongoDB and other OpenStack components. I see two options here:
>>
>> Make sure MongoDB uses dedicated disk(s) on the server where it's
>> installed, and it can be part of existing controller role then
>>
>> Nailgun can make default allocation in a way that MongoDB has dedicated
>> disk by default, if there is more than one disk on the server (which is 100%
>> of real cases, I assume)
>> User's experience would be simply to enable Ceilometer installation by
>> clicking on checkbox. In simple mode, ceilometer + mongo will be installed
>> on controller node. In HA mode, ceilometer + mongo will be installed on all
>> 3 controllers under pacemaker control
>>
>> Make sure MongoDB is installed on a separated server
>>
>> In UI, user will have to:
>>
>> enable ceilometer checkbox ("Install Ceilometer")
>> don't forget to add "ceilometer-db" (mongodb) role to one of the
>> unallocated nodes
>>
>> UI must ensure that this role should not intersect with any other
>> UI must ensure that this role is assigned to at least one node in the env,
>> if ceilometer checkbox is enabled
>> UI must ensure that this role is assigned to at least 3 different
>> unallocated nodes in case of HA deployment mode, to ensure that we will have
>> Ceilometer HA (we can skip this, but add logic that if we have more than one
>> mongo - we must build cluster)
>>
>> In terms of simplicity and ease of use, I would vote for option #1, while
>> leaving ability to place MongoDB on a separate server via Fuel CLI for
>> customized deployments. #1 solves the issue with disk IO by providing
>> dedicated disk(s).
>>
>> > Do we need HA Cluster with non-HA Mongo?
>> For consistency over Fuel story, I vote for HA for all OpenStack
>> components, if HA mode is chosen. So my opinion is no - we do not need such
>> a case.
>>
>> > Puppet manifest are  finished
>> Great! Please pull request them asap - we will need time for reviews. I
>> hope we can complete manifests for HA story by the end of the week too.
>>
>> Thanks,
>>
>>
>> On Mon, Feb 3, 2014 at 5:37 PM, Max Mazur <mmaxur@xxxxxxxxxxxx> wrote:
>>>
>>> Hi!
>>>
>>>
>>> I'd like to add to Fuel the following options:
>>>
>>> 1. Simple install
>>>  - Ceilometer with MongoDB or Ceilometer wit If customer selected Mongo
>>> it is necessary to deploy one more node with MongoDB
>>> Puppet manifest are  finished
>>>
>>> 2. HA Mode
>>>  - Ceilometer with MongoDB replica set.  In this case we need 3 MongoDB
>>> nodes to build HA replica set.
>>> Puppet manifests are in progress now
>>>
>>>
>>> Do we need HA Cluster with non-HA Mongo?
>>>
>>> Best Regards,
>>> Max Mazur
>>>
>>>
>>>
>>> --
>>> Mailing list: https://launchpad.net/~fuel-dev
>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>>
>>
>> --
>> Mike Scherbakov
>> #mihgen
>>
>> --
>> Mailing list: https://launchpad.net/~fuel-dev
>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~fuel-dev
>> More help   : https://help.launchpad.net/ListHelp
>>
>
>
> --
> Mailing list: https://launchpad.net/~fuel-dev
> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~fuel-dev
> More help   : https://help.launchpad.net/ListHelp
>



-- 
Dmitry Borodaenko