openstack team mailing list archive

[Jay Pipes <jaypipes@xxxxxxxxx>]

On 06/29/2012 04:25 AM, Huang Zhiteng wrote:
> Sound like a performance issue.  I think this symptom can be much
> eased if we spend sometime fixing whatever bottleneck causing this
> (slow AMQP, scheduler, or network)?  Now that Nova API has got
> multprocess enabled, we'd move to next bottleneck in long path of
> 'launching instance'.
> Devin, is it possible that you provide more details about this issue
> so that someone else can reproduce it?

Actually, Vish, David Kranz and I had a discussion about similar stuff 
on IRC yesterday. I think that an easy win for this would be to add much 
more fine-grained DEBUG logging statements in the various nova service 
pieces -- nova-compute, nova-network, etc. Right now, there are areas 
that seem to look like performance or locking culprits (iptables 
save/restore for example), but because there isn't very fine-grained 
logging statements, it's tough to say whether:

a) A process (or greenthread) has simply yielded to another while it 
waits for something

b) A process is doing something that is blocking

or

c) A process is doing some other work but no log statements are being 
logged about that work, which makes it seem like some other work is 
taking much longer than it really is

This would be a really easy win for a beginner developer or someone 
looking for something to assist with -- simply add informative 
LOG.debug() statements at various points in the API call pipelines

Best,
-jay