openstack team mailing list archive
-
openstack team
-
Mailing list archive
-
Message #21022
Re: deal with booting lots of instance simultaneously
Are you using multi_host setup? If not, as Vish suggested, that will
alleviate much of the problem.
Best,
-jay
On 02/19/2013 04:09 AM, gtt116 wrote:
> Hi Diego
>
> Thanks for you reply.
> How many hosts do you have? I have 4 hosts. And in this bug,
> https://bugs.launchpad.net/nova/+bug/1094226, The N is 20. In my
> environment N is about 16.
>
> I found that nova-network is too busy to deal with so many rpc request
> at the same time. The Rabbitmq is strong enough in the scenario.
>
> 于 2013年02月19日 16:54, Diego Parrilla Santamaría 写道:
>> Hi gtt,
>>
>> what does it mean for you 'lots of instance simultaneously'? 100,
>> 1000, 10000, more?
>>
>> We have launched >100 (but less than <1000) simultaneously without any
>> issue. Rabbit running in a multicore with several gigs of RAM with out
>> of the box configuration.
>>
>> Cheers
>> Diego
>> --
>> Diego Parrilla
>> <http://www.stackops.com/>*CEO*
>> **www.stackops.com*
>> <http://www.stackops.com/> | * diego.parrilla@xxxxxxxxxxxx
>> <mailto:diego.parrilla@xxxxxxxxxxxx>**** | +34 649 94 43 29
>> | skype:diegoparrilla*
>> * <http://www.stackops.com/>
>> *
>>
>> ****
>>
>> *
>>
>>
>>
>> On Tue, Feb 19, 2013 at 9:35 AM, gtt116 <gtt116@xxxxxxx
>> <mailto:gtt116@xxxxxxx>> wrote:
>>
>> Hi all,
>>
>> When create lots of instance simultaneously, there will be lots of
>> instance in ERROR state. And most of them are caused by network
>> rpc request timeout. This result is not so graceful.
>>
>> I think it will be better if scheduler keep a queue of creating
>> request. when he find all the hosts are busy
>> enough(compute_node.current_workload reach some value), stop cast
>> the request to host temporarily, until he found some host free
>> enough. In this way, we can make sure booting lots of instances
>> simultaneously results in active instances rather than lots of
>> ERROR instance. but will cause a small weak point, if the top
>> value of current_workload small enough, create instance processing
>> will be slow.
>>
>> Do you have another quick fix?
>>
>> Thanks,
>>
>> --
>> best regards,
>> gtt
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> <https://launchpad.net/%7Eopenstack>
>> Post to : openstack@xxxxxxxxxxxxxxxxxxx
>> <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
>> Unsubscribe : https://launchpad.net/~openstack
>> <https://launchpad.net/%7Eopenstack>
>> More help : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
> best regards,
> gtt
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
References