openstack team mailing list archive

["Bret Piatt" <bret@xxxxxxxxxxxxx>]

We can break this down into a list of measureable test cases.  Many of these
tests aren't just single host system tests.  They require an integrated
deployment to find and eliminate the bottleneck.  I'm certain I'm missing
additional items we would want to measure so consider this an off the top of
my head incomplete list.

Rate of change tests:
1. Maximum rate of change per host machine -- this could be create, delete,
migrate, snapshot, backup.
2. Maximum number of host machines per host controller machine that it can
sustain at their maximum rate of change.
3. Maximum number of host machines per glance machine that it can sustain at
their maximum rate of change.
4. Maximum number of requests per second and total buffer of requests on the
message queue per machine.
5. Maximum number of storage volume operations per storage controller
machine.

Scale tests:
1. Maximum number of VMs on a host machine.
2. Maximum number of VMs per host controller.
3. Maximum number of storage volumes attached to a host machine.
4. Maximum number of storage volumes per storage controller machine.
5. Maximum number of VM records in the cloud database.
6. Maximum number of storage volume records in the cloud database.
7. Maximum number of VM images managed per glance machine.

I understand that these maximum numbers will vary greatly depending on what
"a machine", the size of the test VM image, the network configuration, and
many other factors.  I propose we get a test bed setup to measure our own
CloudMIPS (call it whatever we want) like a BogoMIPS so we can measure the
performance of the codebase over time.  We can run the tests each weekend on
the trunk as of 00:00 GMT Saturday and by tracking it against the code
merged each week ensure we're headed in the right direction and if not we'll
be consciously aware of which changes impacted the system performance.  I'm
up for organizing the effort -- finding hardware, DC space, and getting
operational support to manage it.  Is anyone interested in participating?

Back on the AWS example I'll make some sweeping generalizations and round
numbers off to make the math easy.  I'll assume 50k hosts in the measured
facility with a peak rate of change of 150,000/day.  That is only a required
rate of change per host machine of 1 VM / 8 hours.  Even if they only have
12.5k hosts in that facility that is 1 VM / 2 hours per machine.  If they
have 20 VMs sustained then we're looking at 250k to 1M VMs for scale.  

Bret Piatt
OpenStack.org
Twitter: @bpiatt / Mobile 210-867-9338
Open Source Cloud Infrastructure: http://www.openstack.org



-----Original Message-----
From: openstack-bounces+bret=openstack.org@xxxxxxxxxxxxxxxxxxx
[mailto:openstack-bounces+bret=openstack.org@xxxxxxxxxxxxxxxxxxx] On Behalf
Of Ewan Mellor
Sent: Thursday, December 30, 2010 7:48 AM
To: Jay Pipes; openstack@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Openstack] Some insight into the number of instances Nova
needs to spin up...

Those are interesting figures, but it would also be interesting to know how
many of those are kept running.  Our 1M host target is irrelevant if all
those instances are shut down again a day later.  If Amazon are adding *and
keeping* 70k VMs a day then that's a very different matter.

Ewan.

> -----Original Message-----
> From: openstack-bounces+ewan.mellor=citrix.com@xxxxxxxxxxxxxxxxxxx
> [mailto:openstack-bounces+ewan.mellor=citrix.com@xxxxxxxxxxxxxxxxxxx]
> On Behalf Of Jay Pipes
> Sent: 29 December 2010 16:47
> To: openstack@xxxxxxxxxxxxxxxxxxx
> Subject: [Openstack] Some insight into the number of instances Nova 
> needs to spin up...
> 
> Some insight into the number of instances being spun up *per day* on 
> AWS, *just in the US-East-1 region*:
> 
> http://www.jackofallclouds.com/2010/12/recounting-ec2/
> 
> Avg in 2010: 70,528 instances spun up each day
> Max: 150,800 instances in a single day
> 
> Food for thought.
> 
> What do we think Nova could/can support? I know we are aiming at 
> supporting clouds of 1M physical hosts. Perhaps we need to re-assess?
> :)
> 
> /me urges more prioritization of the openstack-ci (continuous 
> integration and performance/stress testing) project in Cactus...
> 
> -jay
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp