openstack team mailing list archive

[Ewan Mellor <Ewan.Mellor@xxxxxxxxxxxxx>]

Yes, I'm definitely interested in participating in this kind of testing effort.  Keep me involved, and I'll bring in other people from Citrix too.

Cheers,

Ewan.

> -----Original Message-----
> From: Bret Piatt [mailto:bret@xxxxxxxxxxxxx]
> Sent: 30 December 2010 16:17
> To: openstack@xxxxxxxxxxxxxxxxxxx
> Cc: 'Jay Pipes'; Ewan Mellor; zaitcev@xxxxxxxxxx; jordan@xxxxxxxxxxxxx
> Subject: RE: [Openstack] Some insight into the number of instances Nova
> needs to spin up...
> 
> We can break this down into a list of measureable test cases.  Many of
> these
> tests aren't just single host system tests.  They require an integrated
> deployment to find and eliminate the bottleneck.  I'm certain I'm
> missing
> additional items we would want to measure so consider this an off the
> top of
> my head incomplete list.
> 
> Rate of change tests:
> 1. Maximum rate of change per host machine -- this could be create,
> delete,
> migrate, snapshot, backup.
> 2. Maximum number of host machines per host controller machine that it
> can
> sustain at their maximum rate of change.
> 3. Maximum number of host machines per glance machine that it can
> sustain at
> their maximum rate of change.
> 4. Maximum number of requests per second and total buffer of requests
> on the
> message queue per machine.
> 5. Maximum number of storage volume operations per storage controller
> machine.
> 
> Scale tests:
> 1. Maximum number of VMs on a host machine.
> 2. Maximum number of VMs per host controller.
> 3. Maximum number of storage volumes attached to a host machine.
> 4. Maximum number of storage volumes per storage controller machine.
> 5. Maximum number of VM records in the cloud database.
> 6. Maximum number of storage volume records in the cloud database.
> 7. Maximum number of VM images managed per glance machine.
> 
> I understand that these maximum numbers will vary greatly depending on
> what
> "a machine", the size of the test VM image, the network configuration,
> and
> many other factors.  I propose we get a test bed setup to measure our
> own
> CloudMIPS (call it whatever we want) like a BogoMIPS so we can measure
> the
> performance of the codebase over time.  We can run the tests each
> weekend on
> the trunk as of 00:00 GMT Saturday and by tracking it against the code
> merged each week ensure we're headed in the right direction and if not
> we'll
> be consciously aware of which changes impacted the system performance.
> I'm
> up for organizing the effort -- finding hardware, DC space, and getting
> operational support to manage it.  Is anyone interested in
> participating?
> 
> Back on the AWS example I'll make some sweeping generalizations and
> round
> numbers off to make the math easy.  I'll assume 50k hosts in the
> measured
> facility with a peak rate of change of 150,000/day.  That is only a
> required
> rate of change per host machine of 1 VM / 8 hours.  Even if they only
> have
> 12.5k hosts in that facility that is 1 VM / 2 hours per machine.  If
> they
> have 20 VMs sustained then we're looking at 250k to 1M VMs for scale.
> 
> Bret Piatt
> OpenStack.org
> Twitter: @bpiatt / Mobile 210-867-9338
> Open Source Cloud Infrastructure: http://www.openstack.org
> 
> 
> 
> -----Original Message-----
> From: openstack-bounces+bret=openstack.org@xxxxxxxxxxxxxxxxxxx
> [mailto:openstack-bounces+bret=openstack.org@xxxxxxxxxxxxxxxxxxx] On
> Behalf
> Of Ewan Mellor
> Sent: Thursday, December 30, 2010 7:48 AM
> To: Jay Pipes; openstack@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Openstack] Some insight into the number of instances Nova
> needs to spin up...
> 
> Those are interesting figures, but it would also be interesting to know
> how
> many of those are kept running.  Our 1M host target is irrelevant if
> all
> those instances are shut down again a day later.  If Amazon are adding
> *and
> keeping* 70k VMs a day then that's a very different matter.
> 
> Ewan.
> 
> > -----Original Message-----
> > From: openstack-bounces+ewan.mellor=citrix.com@xxxxxxxxxxxxxxxxxxx
> > [mailto:openstack-bounces+ewan.mellor=citrix.com@xxxxxxxxxxxxxxxxxxx]
> > On Behalf Of Jay Pipes
> > Sent: 29 December 2010 16:47
> > To: openstack@xxxxxxxxxxxxxxxxxxx
> > Subject: [Openstack] Some insight into the number of instances Nova
> > needs to spin up...
> >
> > Some insight into the number of instances being spun up *per day* on
> > AWS, *just in the US-East-1 region*:
> >
> > http://www.jackofallclouds.com/2010/12/recounting-ec2/
> >
> > Avg in 2010: 70,528 instances spun up each day
> > Max: 150,800 instances in a single day
> >
> > Food for thought.
> >
> > What do we think Nova could/can support? I know we are aiming at
> > supporting clouds of 1M physical hosts. Perhaps we need to re-assess?
> > :)
> >
> > /me urges more prioritization of the openstack-ci (continuous
> > integration and performance/stress testing) project in Cactus...
> >
> > -jay
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~openstack
> > Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp