openstack team mailing list archive

[Steve Heistand <steve.heistand@xxxxxxxx>]

They are all booting the same image. there still may be a random read issues
if they arent mostly synced in copying the file to the compute node.
But there isnt much of a load on the disk while the instances are being launched.
Its just a sata drive at this point so the performance isnt great but better then this.

The network isnt being used much either during launch.  At least as near as I can tell
from netstat and trying to move data around independently of openstack.

scp'ing a file for example from controller to compute (being launched on) shows basically
no different in speed vs no launch on at the same time.

I did have to drop the mtu9000 on the network devices. Not sure if it was the switch or what
but after so much data performance just tanked and stayed bad. on normal mtu1500 it stayed
good.


On 04/30/2013 11:56 AM, Rick Jones wrote:
> On 04/30/2013 11:42 AM, Steve Heistand wrote:
>> if I launch one vm at a time its doesnt very long to start up the instance. maybe a minute.
>> if I launch 4 instances (of the same snapshot as before) it takes 30 minutes.
>>
>> they are all launching to different compute nodes, the controllers are all multicore,
>> I dont see any processes on the compute nodes taking much cpu power, the controller has
>> a keystone process mostly sucking up 1 core, loads and loads of beam.smp from rabbitmq but
>> none are really taking any cpu time.
>>
>> the glance image storage is on the controller node, the snapshots are 2-3G in size.
>>
>> where should I start looking to find out why things are so slow?
> 
> I am something of a "networking guy" so that will color my response :)
> 
> Is each  instance using the same image or a different one?  If a 
> different one, the "workload" to the glance storage will become (I 
> suspect) a random read workload accessing the four different images even 
> though each individual stream may be sequential.  How much "storage 
> oomph" do you have on/serving the glance/controller node?
> 
> After that, what do the network statistics look like?  Starting I 
> suppose with the glance/controller node.  Take some snapshots over an 
> interval in each case and run them through something like beforeafter:
> 
> netstat -s > before
> ...wait a defined/consistent moment...
> netstat -s > after
> beforeafter before after > delta
> 
> and go from there.
> 
> rick jones
> 

-- 
************************************************************************
 Steve Heistand                           NASA Ames Research Center
 SciCon Group                             Mail Stop 258-6
 steve.heistand@xxxxxxxx  (650) 604-4369  Moffett Field, CA 94035-1000
************************************************************************
 "Any opinions expressed are those of our alien overlords, not my own."

# For Remedy                        #
#Action: Resolve                    #
#Resolution: Resolved               #
#Reason: No Further Action Required #
#Tier1: User Code                   #
#Tier2: Other                       #
#Tier3: Assistance                  #
#Notification: None                 #

Attachment: signature.asc
Description: OpenPGP digital signature