← Back to team overview

fuel-dev team mailing list archive

Re: Maximum number of controllers & adding controllers

 

So where did we leave off on this discussion?

I see that https://review.openstack.org/#/c/73546/ was merged even
though there still was discussion. When nailgun sees any controller in
the list, it redeploys them all. I don't see why we needed to disable
deploying controllers later.

On Wed, Feb 19, 2014 at 12:36 AM, Evgeniy L <eli@xxxxxxxxxxxx> wrote:
> Hi,
>
> In case of ha we redeploy all controllers [0] if we need to redeploy at
> least one controller.
> So in case of failed controller or if new controller was added we redeploy
> all of them.
>
> [0]
> https://github.com/stackforge/fuel-web/blob/stable/4.0/nailgun/nailgun/task/helpers.py#L369-L394
>
>
>
> On Tue, Feb 18, 2014 at 8:12 PM, Vladimir Kuklin <vkuklin@xxxxxxxxxxxx>
> wrote:
>>
>> I would ask our Nailgun guys about detailed description on how we are
>> handling nodes addition. I am not quite sure if adding of controller
>> triggers redeployment of controllers all the time if it does not break the
>> installation. AFAIK, we still do not have any test cases for this. Thus, it
>> is the question to our Nailgun and QA teams.
>>
>> Nastya, Dmitry, Evgeniy could you please comment?
>>
>>
>> On Tue, Feb 18, 2014 at 7:45 PM, David Easter <deaster@xxxxxxxxxxxx>
>> wrote:
>>>
>>> It did take a long time (at least an hour) to redeploy the other
>>> controllers.  I didn't see it redeploy the compute node(s) so I think those
>>> were untouched.
>>>
>>> What I'm pointing out is that based on my (simple) test, it looks like we
>>> can add a controller to an existing environment.  Fuel 4.0 redeploys all the
>>> controllers when that a new controller is added.  While inefficient, it does
>>> appear to be accomplishing the task.   If this is indeed the case, I'd say
>>> to leave the capability in the project for now and we'll improve upon it in
>>> 5.0.
>>>
>>> Thanks,
>>>
>>> -Dave Easter
>>>
>>> From: Vladimir Kuklin <vkuklin@xxxxxxxxxxxx>
>>> Date: Tuesday, February 18, 2014 at 7:33 AM
>>> To: David Easter <deaster@xxxxxxxxxxxx>
>>> Cc: Mike Scherbakov <mscherbakov@xxxxxxxxxxxx>, Anastasia Urlapova
>>> <aurlapova@xxxxxxxxxxxx>, Vitaly Kramskikh <vkramskikh@xxxxxxxxxxxx>, Julia
>>> Aranovich <jkirnosova@xxxxxxxxxxxx>, Roman Alekseenkov
>>> <ralekseenkov@xxxxxxxxxxxx>, Nikolay Markov <nmarkov@xxxxxxxxxxxx>, Vladimir
>>> Sharshov <vsharshov@xxxxxxxxxxxx>, Bogdan Dudko <bdudko@xxxxxxxxxxxx>, Meg
>>> McRoberts <mmcroberts@xxxxxxxxxxxx>, "fuel-dev@xxxxxxxxxxxxxxxxxxx"
>>> <fuel-dev@xxxxxxxxxxxxxxxxxxx>
>>>
>>> Subject: Re: Maximum number of controllers & adding controllers
>>>
>>> If you redeploy the whole environment after controller is added, that
>>> should work, but it looks really weird and takes a lot of time. Currently it
>>> is constrained by our orchestration finite-state-machine that marks node as
>>> ready after the deployment and does not redeploy it after cluster is
>>> changed. We are going to make it 200% more flexible in the future releases,
>>> thus allowing us to do real lifecycle management including massive cluster
>>> transformation and even upgrades of major OpenStack releases.
>>>
>>>
>>> On Tue, Feb 18, 2014 at 6:40 PM, David Easter <deaster@xxxxxxxxxxxx>
>>> wrote:
>>>>
>>>> So just to confirm - even though the UI enables the user to add an
>>>> additional controller, Fuel does successfully add the controller and marks
>>>> all 4 controllers as "ready", the stack won't work properly?   So the defect
>>>> is that Fuel shouldn't enable a user to add additional controllers, but
>>>> we're looking to correct this in 5.0?
>>>>
>>>> So in 4.0, the redeploying of all the other controllers when one was
>>>> added wasn't doing the work to fix the configuration for rabbitmq, haproxy,
>>>> myswl, etc.?
>>>>
>>>> Thanks,
>>>>
>>>> -Dave Easter
>>>>
>>>> From: Mike Scherbakov <mscherbakov@xxxxxxxxxxxx>
>>>> Date: Tuesday, February 18, 2014 at 3:06 AM
>>>> To: Vladimir Kuklin <vkuklin@xxxxxxxxxxxx>, Anastasia Urlapova
>>>> <aurlapova@xxxxxxxxxxxx>
>>>> Cc: David Easter <deaster@xxxxxxxxxxxx>, Vitaly Kramskikh
>>>> <vkramskikh@xxxxxxxxxxxx>, Julia Aranovich <jkirnosova@xxxxxxxxxxxx>, Roman
>>>> Alekseenkov <ralekseenkov@xxxxxxxxxxxx>, Nikolay Markov
>>>> <nmarkov@xxxxxxxxxxxx>, Vladimir Sharshov <vsharshov@xxxxxxxxxxxx>, Bogdan
>>>> Dudko <bdudko@xxxxxxxxxxxx>, Meg McRoberts <mmcroberts@xxxxxxxxxxxx>,
>>>> "fuel-dev@xxxxxxxxxxxxxxxxxxx" <fuel-dev@xxxxxxxxxxxxxxxxxxx>
>>>> Subject: Re: Maximum number of controllers & adding controllers
>>>>
>>>> I chatted with Vladimir in #fuel-dev, and clarified the situation.
>>>> So, for my point a) or #2 in Vladimir's response - when we add new
>>>> controllers to deployed env - we need to provide UI message and prevent user
>>>> from doing so, as it is not supported at the moment.
>>>>
>>>> For creating new envs with >3 controllers - it should work in theory.
>>>> Nastya, waiting from you results of testing of this case.
>>>>
>>>> Thanks,
>>>>
>>>>
>>>> On Tue, Feb 18, 2014 at 1:11 PM, Vladimir Kuklin <vkuklin@xxxxxxxxxxxx>
>>>> wrote:
>>>>>
>>>>> Mike
>>>>>
>>>>> 1) I am not aware of any problems of initial N-controller setup
>>>>> configuration. Could anyone point me to the bug regarding this problem?
>>>>> 2) Adding of the controller is the same problem as I already pointed -
>>>>> it is related to the cluster lifecycle management which we do not have right
>>>>> now and are really striving hard to (partially-) implement in 5.0 release.
>>>>> As soon as we have controller added - we need to go through all the
>>>>> controllers and fix configuration for: rabbitmq, haproxy, mysql and so on.
>>>>> Thus, we need to split deployment into granular roles and re-run deployment
>>>>> for particular roles. Currently, our roles are far too huge to do this,
>>>>> that's why we need granular deployment feature.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Feb 18, 2014 at 12:54 PM, Mike Scherbakov
>>>>> <mscherbakov@xxxxxxxxxxxx> wrote:
>>>>>>
>>>>>> Fuelers, as far as know, deployment or operation breaks if:
>>>>>> a) we add controllers to existing deployed environment (let's say we
>>>>>> had 1, then we add another one in attempt to make it HAed)
>>>>>> b) we add more than 3 controllers in HA mode, regardless of if 3 were
>>>>>> deployed and we add 1 new, or we try to initially deploy 4.
>>>>>> And we had this limitation for the very beginning, our UI just was not
>>>>>> handling this situation - so now we are fixing UI behavior.
>>>>>>
>>>>>> Vladimir, can you please update with latest status on this, and
>>>>>> provide short info on why we have such limitations if it is the case?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 17, 2014 at 11:09 PM, David Easter <deaster@xxxxxxxxxxxx>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Vitaly,
>>>>>>>
>>>>>>>    Expanding to fuel-dev for discussion.  I'm a bit confused on where
>>>>>>> those messages would be displayed.  I brought up an environment deployed by
>>>>>>> Fuel 4.0 in HA mode (3 controllers) and added a fourth controller.  I did
>>>>>>> not get an error message.  So it looks like (a) Fuel can deploy more than 3
>>>>>>> controllers in an HA environment and (b) that I can add an additional
>>>>>>> controller to an already deployed cluster.  In adding the fourth controller
>>>>>>> to the HA environment, it looks like it redeployed the other  controllers as
>>>>>>> well  - but still was successful.
>>>>>>>
>>>>>>>   Perhaps these are outdated messages?
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> - David J. Easter
>>>>>>>   Product Line Manager,  Mirantis
>>>>>>>>
>>>>>>>>
>>>>>>>> From: Vitaly Kramskikh <vkramskikh@xxxxxxxxxxxx>
>>>>>>>> Date: Monday, February 17, 2014 at 6:30 AM
>>>>>>>> To: David Easter <deaster@xxxxxxxxxxxx>
>>>>>>>> Cc: Julia Aranovich <jkirnosova@xxxxxxxxxxxx>, Mike Scherbakov
>>>>>>>> <mscherbakov@xxxxxxxxxxxx>, Roman Alekseenkov <ralekseenkov@xxxxxxxxxxxx>,
>>>>>>>> Nikolay Markov <nmarkov@xxxxxxxxxxxx>, Vladimir Sharshov
>>>>>>>> <vsharshov@xxxxxxxxxxxx>, Bogdan Dudko <bdudko@xxxxxxxxxxxx>,
>>>>>>>> <mmcroberts@xxxxxxxxxxxx>
>>>>>>>>
>>>>>>>> Subject: Re: Need translations for Stop Deployment && Reset
>>>>>>>> environment features on UI
>>>>>>>>
>>>>>>>> Hi David,
>>>>>>>>
>>>>>>>> Could you please also propose better translations for:
>>>>>>>> "New controllers cannot be added to an operational environment."
>>>>>>>> (shown when user tries to add more controllers to already deployed cluster)
>>>>>>>> "Only 3 controllers can be assigned in a Highly-Available
>>>>>>>> environment. " (shown when user tries to add more than 3 controllers to HA
>>>>>>>> cluster)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Mike Scherbakov
>>>>>> #mihgen
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Yours Faithfully,
>>>>> Vladimir Kuklin,
>>>>> Senior Deployment Engineer,
>>>>> Mirantis, Inc.
>>>>> +7 (495) 640-49-04
>>>>> +7 (926) 702-39-68
>>>>> Skype kuklinvv
>>>>> 45bk3, Vorontsovskaya Str.
>>>>> Moscow, Russia,
>>>>> www.mirantis.com
>>>>> www.mirantis.ru
>>>>> vkuklin@xxxxxxxxxxxx
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Mike Scherbakov
>>>> #mihgen
>>>
>>>
>>>
>>>
>>> --
>>> Yours Faithfully,
>>> Vladimir Kuklin,
>>> Senior Deployment Engineer,
>>> Mirantis, Inc.
>>> +7 (495) 640-49-04
>>> +7 (926) 702-39-68
>>> Skype kuklinvv
>>> 45bk3, Vorontsovskaya Str.
>>> Moscow, Russia,
>>> www.mirantis.com
>>> www.mirantis.ru
>>> vkuklin@xxxxxxxxxxxx
>>
>>
>>
>>
>> --
>> Yours Faithfully,
>> Vladimir Kuklin,
>> Senior Deployment Engineer,
>> Mirantis, Inc.
>> +7 (495) 640-49-04
>> +7 (926) 702-39-68
>> Skype kuklinvv
>> 45bk3, Vorontsovskaya Str.
>> Moscow, Russia,
>> www.mirantis.com
>> www.mirantis.ru
>> vkuklin@xxxxxxxxxxxx
>
>
>
> --
> Mailing list: https://launchpad.net/~fuel-dev
> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~fuel-dev
> More help   : https://help.launchpad.net/ListHelp
>



-- 
If google has done it, Google did it right!


Follow ups

References