openstack team mailing list archive
-
openstack team
-
Mailing list archive
-
Message #21015
Re: deal with booting lots of instance simultaneously
Hi Diego
Thanks for you reply.
How many hosts do you have? I have 4 hosts. And in this bug,
https://bugs.launchpad.net/nova/+bug/1094226, The N is 20. In my
environment N is about 16.
I found that nova-network is too busy to deal with so many rpc request
at the same time. The Rabbitmq is strong enough in the scenario.
? 2013?02?19? 16:54, Diego Parrilla Santamaría ??:
> Hi gtt,
>
> what does it mean for you 'lots of instance simultaneously'? 100,
> 1000, 10000, more?
>
> We have launched >100 (but less than <1000) simultaneously without any
> issue. Rabbit running in a multicore with several gigs of RAM with out
> of the box configuration.
>
> Cheers
> Diego
> --
> Diego Parrilla
> <http://www.stackops.com/>*CEO*
> **www.stackops.com*
> <http://www.stackops.com/> | * diego.parrilla@xxxxxxxxxxxx
> <mailto:diego.parrilla@xxxxxxxxxxxx>**** | +34 649 94 43 29
> | skype:diegoparrilla*
> * <http://www.stackops.com/>
> *
>
> ****
>
> *
>
>
>
> On Tue, Feb 19, 2013 at 9:35 AM, gtt116 <gtt116@xxxxxxx
> <mailto:gtt116@xxxxxxx>> wrote:
>
> Hi all,
>
> When create lots of instance simultaneously, there will be lots of
> instance in ERROR state. And most of them are caused by network
> rpc request timeout. This result is not so graceful.
>
> I think it will be better if scheduler keep a queue of creating
> request. when he find all the hosts are busy
> enough(compute_node.current_workload reach some value), stop cast
> the request to host temporarily, until he found some host free
> enough. In this way, we can make sure booting lots of instances
> simultaneously results in active instances rather than lots of
> ERROR instance. but will cause a small weak point, if the top
> value of current_workload small enough, create instance processing
> will be slow.
>
> Do you have another quick fix?
>
> Thanks,
>
> --
> best regards,
> gtt
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> <https://launchpad.net/%7Eopenstack>
> Post to : openstack@xxxxxxxxxxxxxxxxxxx
> <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> Unsubscribe : https://launchpad.net/~openstack
> <https://launchpad.net/%7Eopenstack>
> More help : https://help.launchpad.net/ListHelp
>
>
--
best regards,
gtt
Follow ups
References