fuel-dev team mailing list archive

[Mike Scherbakov <mscherbakov@xxxxxxxxxxxx>]

Thanks to #dmit2k, it was discovered, that now we can't even add
controllers if deployment finished with error, and you need to remove
failed node and *add* new one.

We decided to simply revert https://review.openstack.org/#/c/76982/.


On Thu, Feb 27, 2014 at 2:00 PM, Mike Scherbakov
<mscherbakov@xxxxxxxxxxxx>wrote:

> Thanks for reminding, Andrew.
>
> I did not hear back from Nastya about testing the case of adding more
> controllers in HA in already deployed env.
> If it works, even with redeployment of all, than we can provide a patch to
> stable/4.1 with this, I assume.
>
> Thanks,
>
>
> On Thu, Feb 27, 2014 at 2:27 AM, Andrew Woodward <xarses@xxxxxxxxx> wrote:
>
>> So where did we leave off on this discussion?
>>
>> I see that https://review.openstack.org/#/c/73546/ was merged even
>> though there still was discussion. When nailgun sees any controller in
>> the list, it redeploys them all. I don't see why we needed to disable
>> deploying controllers later.
>>
>> On Wed, Feb 19, 2014 at 12:36 AM, Evgeniy L <eli@xxxxxxxxxxxx> wrote:
>> > Hi,
>> >
>> > In case of ha we redeploy all controllers [0] if we need to redeploy at
>> > least one controller.
>> > So in case of failed controller or if new controller was added we
>> redeploy
>> > all of them.
>> >
>> > [0]
>> >
>> https://github.com/stackforge/fuel-web/blob/stable/4.0/nailgun/nailgun/task/helpers.py#L369-L394
>> >
>> >
>> >
>> > On Tue, Feb 18, 2014 at 8:12 PM, Vladimir Kuklin <vkuklin@xxxxxxxxxxxx>
>> > wrote:
>> >>
>> >> I would ask our Nailgun guys about detailed description on how we are
>> >> handling nodes addition. I am not quite sure if adding of controller
>> >> triggers redeployment of controllers all the time if it does not break
>> the
>> >> installation. AFAIK, we still do not have any test cases for this.
>> Thus, it
>> >> is the question to our Nailgun and QA teams.
>> >>
>> >> Nastya, Dmitry, Evgeniy could you please comment?
>> >>
>> >>
>> >> On Tue, Feb 18, 2014 at 7:45 PM, David Easter <deaster@xxxxxxxxxxxx>
>> >> wrote:
>> >>>
>> >>> It did take a long time (at least an hour) to redeploy the other
>> >>> controllers.  I didn't see it redeploy the compute node(s) so I think
>> those
>> >>> were untouched.
>> >>>
>> >>> What I'm pointing out is that based on my (simple) test, it looks
>> like we
>> >>> can add a controller to an existing environment.  Fuel 4.0 redeploys
>> all the
>> >>> controllers when that a new controller is added.  While inefficient,
>> it does
>> >>> appear to be accomplishing the task.   If this is indeed the case,
>> I'd say
>> >>> to leave the capability in the project for now and we'll improve upon
>> it in
>> >>> 5.0.
>> >>>
>> >>> Thanks,
>> >>>
>> >>> -Dave Easter
>> >>>
>> >>> From: Vladimir Kuklin <vkuklin@xxxxxxxxxxxx>
>> >>> Date: Tuesday, February 18, 2014 at 7:33 AM
>> >>> To: David Easter <deaster@xxxxxxxxxxxx>
>> >>> Cc: Mike Scherbakov <mscherbakov@xxxxxxxxxxxx>, Anastasia Urlapova
>> >>> <aurlapova@xxxxxxxxxxxx>, Vitaly Kramskikh <vkramskikh@xxxxxxxxxxxx>,
>> Julia
>> >>> Aranovich <jkirnosova@xxxxxxxxxxxx>, Roman Alekseenkov
>> >>> <ralekseenkov@xxxxxxxxxxxx>, Nikolay Markov <nmarkov@xxxxxxxxxxxx>,
>> Vladimir
>> >>> Sharshov <vsharshov@xxxxxxxxxxxx>, Bogdan Dudko <bdudko@xxxxxxxxxxxx>,
>> Meg
>> >>> McRoberts <mmcroberts@xxxxxxxxxxxx>, "fuel-dev@xxxxxxxxxxxxxxxxxxx"
>> >>> <fuel-dev@xxxxxxxxxxxxxxxxxxx>
>> >>>
>> >>> Subject: Re: Maximum number of controllers & adding controllers
>> >>>
>> >>> If you redeploy the whole environment after controller is added, that
>> >>> should work, but it looks really weird and takes a lot of time.
>> Currently it
>> >>> is constrained by our orchestration finite-state-machine that marks
>> node as
>> >>> ready after the deployment and does not redeploy it after cluster is
>> >>> changed. We are going to make it 200% more flexible in the future
>> releases,
>> >>> thus allowing us to do real lifecycle management including massive
>> cluster
>> >>> transformation and even upgrades of major OpenStack releases.
>> >>>
>> >>>
>> >>> On Tue, Feb 18, 2014 at 6:40 PM, David Easter <deaster@xxxxxxxxxxxx>
>> >>> wrote:
>> >>>>
>> >>>> So just to confirm - even though the UI enables the user to add an
>> >>>> additional controller, Fuel does successfully add the controller and
>> marks
>> >>>> all 4 controllers as "ready", the stack won't work properly?   So
>> the defect
>> >>>> is that Fuel shouldn't enable a user to add additional controllers,
>> but
>> >>>> we're looking to correct this in 5.0?
>> >>>>
>> >>>> So in 4.0, the redeploying of all the other controllers when one was
>> >>>> added wasn't doing the work to fix the configuration for rabbitmq,
>> haproxy,
>> >>>> myswl, etc.?
>> >>>>
>> >>>> Thanks,
>> >>>>
>> >>>> -Dave Easter
>> >>>>
>> >>>> From: Mike Scherbakov <mscherbakov@xxxxxxxxxxxx>
>> >>>> Date: Tuesday, February 18, 2014 at 3:06 AM
>> >>>> To: Vladimir Kuklin <vkuklin@xxxxxxxxxxxx>, Anastasia Urlapova
>> >>>> <aurlapova@xxxxxxxxxxxx>
>> >>>> Cc: David Easter <deaster@xxxxxxxxxxxx>, Vitaly Kramskikh
>> >>>> <vkramskikh@xxxxxxxxxxxx>, Julia Aranovich <jkirnosova@xxxxxxxxxxxx>,
>> Roman
>> >>>> Alekseenkov <ralekseenkov@xxxxxxxxxxxx>, Nikolay Markov
>> >>>> <nmarkov@xxxxxxxxxxxx>, Vladimir Sharshov <vsharshov@xxxxxxxxxxxx>,
>> Bogdan
>> >>>> Dudko <bdudko@xxxxxxxxxxxx>, Meg McRoberts <mmcroberts@xxxxxxxxxxxx
>> >,
>> >>>> "fuel-dev@xxxxxxxxxxxxxxxxxxx" <fuel-dev@xxxxxxxxxxxxxxxxxxx>
>> >>>> Subject: Re: Maximum number of controllers & adding controllers
>> >>>>
>> >>>> I chatted with Vladimir in #fuel-dev, and clarified the situation.
>> >>>> So, for my point a) or #2 in Vladimir's response - when we add new
>> >>>> controllers to deployed env - we need to provide UI message and
>> prevent user
>> >>>> from doing so, as it is not supported at the moment.
>> >>>>
>> >>>> For creating new envs with >3 controllers - it should work in theory.
>> >>>> Nastya, waiting from you results of testing of this case.
>> >>>>
>> >>>> Thanks,
>> >>>>
>> >>>>
>> >>>> On Tue, Feb 18, 2014 at 1:11 PM, Vladimir Kuklin <
>> vkuklin@xxxxxxxxxxxx>
>> >>>> wrote:
>> >>>>>
>> >>>>> Mike
>> >>>>>
>> >>>>> 1) I am not aware of any problems of initial N-controller setup
>> >>>>> configuration. Could anyone point me to the bug regarding this
>> problem?
>> >>>>> 2) Adding of the controller is the same problem as I already
>> pointed -
>> >>>>> it is related to the cluster lifecycle management which we do not
>> have right
>> >>>>> now and are really striving hard to (partially-) implement in 5.0
>> release.
>> >>>>> As soon as we have controller added - we need to go through all the
>> >>>>> controllers and fix configuration for: rabbitmq, haproxy, mysql and
>> so on.
>> >>>>> Thus, we need to split deployment into granular roles and re-run
>> deployment
>> >>>>> for particular roles. Currently, our roles are far too huge to do
>> this,
>> >>>>> that's why we need granular deployment feature.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Tue, Feb 18, 2014 at 12:54 PM, Mike Scherbakov
>> >>>>> <mscherbakov@xxxxxxxxxxxx> wrote:
>> >>>>>>
>> >>>>>> Fuelers, as far as know, deployment or operation breaks if:
>> >>>>>> a) we add controllers to existing deployed environment (let's say
>> we
>> >>>>>> had 1, then we add another one in attempt to make it HAed)
>> >>>>>> b) we add more than 3 controllers in HA mode, regardless of if 3
>> were
>> >>>>>> deployed and we add 1 new, or we try to initially deploy 4.
>> >>>>>> And we had this limitation for the very beginning, our UI just was
>> not
>> >>>>>> handling this situation - so now we are fixing UI behavior.
>> >>>>>>
>> >>>>>> Vladimir, can you please update with latest status on this, and
>> >>>>>> provide short info on why we have such limitations if it is the
>> case?
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>>
>> >>>>>>
>> >>>>>> On Mon, Feb 17, 2014 at 11:09 PM, David Easter <
>> deaster@xxxxxxxxxxxx>
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>> Hi Vitaly,
>> >>>>>>>
>> >>>>>>>    Expanding to fuel-dev for discussion.  I'm a bit confused on
>> where
>> >>>>>>> those messages would be displayed.  I brought up an environment
>> deployed by
>> >>>>>>> Fuel 4.0 in HA mode (3 controllers) and added a fourth
>> controller.  I did
>> >>>>>>> not get an error message.  So it looks like (a) Fuel can deploy
>> more than 3
>> >>>>>>> controllers in an HA environment and (b) that I can add an
>> additional
>> >>>>>>> controller to an already deployed cluster.  In adding the fourth
>> controller
>> >>>>>>> to the HA environment, it looks like it redeployed the other
>>  controllers as
>> >>>>>>> well  - but still was successful.
>> >>>>>>>
>> >>>>>>>   Perhaps these are outdated messages?
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>>
>> >>>>>>> - David J. Easter
>> >>>>>>>   Product Line Manager,  Mirantis
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> From: Vitaly Kramskikh <vkramskikh@xxxxxxxxxxxx>
>> >>>>>>>> Date: Monday, February 17, 2014 at 6:30 AM
>> >>>>>>>> To: David Easter <deaster@xxxxxxxxxxxx>
>> >>>>>>>> Cc: Julia Aranovich <jkirnosova@xxxxxxxxxxxx>, Mike Scherbakov
>> >>>>>>>> <mscherbakov@xxxxxxxxxxxx>, Roman Alekseenkov <
>> ralekseenkov@xxxxxxxxxxxx>,
>> >>>>>>>> Nikolay Markov <nmarkov@xxxxxxxxxxxx>, Vladimir Sharshov
>> >>>>>>>> <vsharshov@xxxxxxxxxxxx>, Bogdan Dudko <bdudko@xxxxxxxxxxxx>,
>> >>>>>>>> <mmcroberts@xxxxxxxxxxxx>
>> >>>>>>>>
>> >>>>>>>> Subject: Re: Need translations for Stop Deployment && Reset
>> >>>>>>>> environment features on UI
>> >>>>>>>>
>> >>>>>>>> Hi David,
>> >>>>>>>>
>> >>>>>>>> Could you please also propose better translations for:
>> >>>>>>>> "New controllers cannot be added to an operational environment."
>> >>>>>>>> (shown when user tries to add more controllers to already
>> deployed cluster)
>> >>>>>>>> "Only 3 controllers can be assigned in a Highly-Available
>> >>>>>>>> environment. " (shown when user tries to add more than 3
>> controllers to HA
>> >>>>>>>> cluster)
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> Mike Scherbakov
>> >>>>>> #mihgen
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Yours Faithfully,
>> >>>>> Vladimir Kuklin,
>> >>>>> Senior Deployment Engineer,
>> >>>>> Mirantis, Inc.
>> >>>>> +7 (495) 640-49-04
>> >>>>> +7 (926) 702-39-68
>> >>>>> Skype kuklinvv
>> >>>>> 45bk3, Vorontsovskaya Str.
>> >>>>> Moscow, Russia,
>> >>>>> www.mirantis.com
>> >>>>> www.mirantis.ru
>> >>>>> vkuklin@xxxxxxxxxxxx
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Mike Scherbakov
>> >>>> #mihgen
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Yours Faithfully,
>> >>> Vladimir Kuklin,
>> >>> Senior Deployment Engineer,
>> >>> Mirantis, Inc.
>> >>> +7 (495) 640-49-04
>> >>> +7 (926) 702-39-68
>> >>> Skype kuklinvv
>> >>> 45bk3, Vorontsovskaya Str.
>> >>> Moscow, Russia,
>> >>> www.mirantis.com
>> >>> www.mirantis.ru
>> >>> vkuklin@xxxxxxxxxxxx
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Yours Faithfully,
>> >> Vladimir Kuklin,
>> >> Senior Deployment Engineer,
>> >> Mirantis, Inc.
>> >> +7 (495) 640-49-04
>> >> +7 (926) 702-39-68
>> >> Skype kuklinvv
>> >> 45bk3, Vorontsovskaya Str.
>> >> Moscow, Russia,
>> >> www.mirantis.com
>> >> www.mirantis.ru
>> >> vkuklin@xxxxxxxxxxxx
>> >
>> >
>> >
>> > --
>> > Mailing list: https://launchpad.net/~fuel-dev
>> > Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>> > Unsubscribe : https://launchpad.net/~fuel-dev
>> > More help   : https://help.launchpad.net/ListHelp
>> >
>>
>>
>>
>> --
>> If google has done it, Google did it right!
>>
>> --
>> Mailing list: https://launchpad.net/~fuel-dev
>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~fuel-dev
>> More help   : https://help.launchpad.net/ListHelp
>>
>
>
>
> --
> Mike Scherbakov
> #mihgen
>



-- 
Mike Scherbakov
#mihgen