← Back to team overview

openstack team mailing list archive

Re: Nova and asynchronous instance launching

 

I'm not expecting a client to do anything, and I'm not sure where you got that from my response below... I'm talking about adding debug statements into the nova-compute/nova-network logs that an *operator* or *core developer* would use to determine which parts of the code are taking that most amount of time.

-jay

On 06/29/2012 05:45 PM, Doug Davis wrote:

You don't really expect a client (think ec2-like-user) to analyze debug
info do you?

I really think we need a nice consistent way for people to see what's
going on with long-running operations.  Debug info isn't that to me.

thanks
-Doug
______________________________________________________
STSM |  Standards Architect  |  IBM Software Group
(919) 254-6905  |  IBM 444-6905  |  dug@xxxxxxxxxx
The more I'm around some people, the more I like my dog.


*Jay Pipes <jaypipes@xxxxxxxxx>*
Sent by: openstack-bounces+dug=us.ibm.com@xxxxxxxxxxxxxxxxxxx

06/29/2012 01:46 PM

	
To
	Huang Zhiteng <winston.d@xxxxxxxxx>
cc
	openstack@xxxxxxxxxxxxxxxxxxx
Subject
	Re: [Openstack] Nova and asynchronous instance launching


	





On 06/29/2012 04:25 AM, Huang Zhiteng wrote:
 > Sound like a performance issue.  I think this symptom can be much
 > eased if we spend sometime fixing whatever bottleneck causing this
 > (slow AMQP, scheduler, or network)?  Now that Nova API has got
 > multprocess enabled, we'd move to next bottleneck in long path of
 > 'launching instance'.
 > Devin, is it possible that you provide more details about this issue
 > so that someone else can reproduce it?

Actually, Vish, David Kranz and I had a discussion about similar stuff
on IRC yesterday. I think that an easy win for this would be to add much
more fine-grained DEBUG logging statements in the various nova service
pieces -- nova-compute, nova-network, etc. Right now, there are areas
that seem to look like performance or locking culprits (iptables
save/restore for example), but because there isn't very fine-grained
logging statements, it's tough to say whether:

a) A process (or greenthread) has simply yielded to another while it
waits for something

b) A process is doing something that is blocking

or

c) A process is doing some other work but no log statements are being
logged about that work, which makes it seem like some other work is
taking much longer than it really is

This would be a really easy win for a beginner developer or someone
looking for something to assist with -- simply add informative
LOG.debug() statements at various points in the API call pipelines

Best,
-jay

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp





Follow ups

References