openstack team mailing list archive

[Kapil Thangavelu <kapil.thangavelu@xxxxxxxxxxxxx>]

Excerpts from Monsyne Dragon's message of 2012-03-02 16:10:01 -0500:
> 
> On Mar 2, 2012, at 9:17 AM, Jay Pipes wrote:
> 
> > On 03/02/2012 05:34 AM, Day, Phil wrote:
> >> In our experience (running clusters of several hundred nodes) the DB performance is not generally the significant factor, so making its calls non-blocking  gives only a very small increase in processing capacity and creates other side effects in terms of slowing all eventlets down as they wait for their turn to run.
> > 
> > Yes, I believe I said that this was the case at the last design summit -- or rather, I believe I said "is there any evidence that the database is a performance or scalability problem at all"?
> > 
> >> That shouldn't really be surprising given that the Nova DB is pretty small and MySQL is a pretty good DB - throw reasonable hardware at the DB server and give it a bit of TLC from a DBA (remove deleted entries from the DB, add indexes where the slow query log tells you to, etc) and it shouldn't be the bottleneck in the system for performance or scalability.
> > 
> > ++
> > 
> >> We use the python driver and have experimented with allowing the eventlet code to make the db calls non-blocking (its not the default setting), and it works, but didn't give us any significant advantage.
> > 
> > Yep, identical results to the work that Mark Washenberger did on the same subject.
> > 
> 
> Has anyone thought about switching to gevent?   It's similar enough to eventlet that the port shouldn't be too bad, and because it's event loop is in C, (libevent), there are C mysql drivers (ultramysql) that will work with it without blocking.   


Switching to gevent won't fix the structural problems with the codebase, that 
nescitated sleeps for greenlet switching. A refactoring to an architecture more 
amenable to decomposing api requests into discrete tasks executed that are 
yieldable would help. incidentally, ultramysql is not dbapi compliant, and won't 
work with sqlalchemy.

-kapil