openstack team mailing list archive

[Monty Taylor <mordred@xxxxxxxxxxxx>]

Hrm. It's not piling on with the locking like I originally thought -
from the db's perspective there's not a whole hell of a lot going on,
actually - even two copies of this script run concurrently cause them to
totally get in to some app-level spinning that ends up with at least one
script erroring. I'll keep poking, mainly just because it's interesting
- but it's certainly not happy.

On 09/25/2011 01:53 PM, Vishvananda Ishaya wrote:
> Hey everyone,
> 
> I'm a bit concerned with the connection pooling in the db.  It seems that things are not getting cleaned up properly.  I have a repro-case that causes failures that we have seen before.  if I revert the nova/db/sqlalchemy/session.py to before the eventlet db pool was added I get no failures.  If you want to see the issue, try the attached code.  You will need to run from the nova directory or do python setup.py develop.  You will also need to create a mysql database called test and edit the sql_connection string if you have a mysql password, etc.  Please check this code.  If we can't come up with a fix, I htink we need to revert back to no connection pooling.
> 
> Run the attached script at least 3 times The code below runs fine the first couple of times, Then it starts to fail with the following error:
> 
> 2011-09-24 12:51:02,799 INFO sqlalchemy.engine.base.Engine.0x...36d0 [-] ROLLBACK
> Traceback (most recent call last):
>  File "/Library/Python/2.7/site-packages/eventlet/hubs/hub.py", line 336, in fire_timers
>    timer()
>  File "/Library/Python/2.7/site-packages/eventlet/hubs/timer.py", line 56, in __call__
>    cb(*args, **kw)
>  File "/Library/Python/2.7/site-packages/eventlet/event.py", line 163, in _do_send
>    waiter.switch(result)
>  File "/Library/Python/2.7/site-packages/eventlet/greenthread.py", line 192, in main
>    result = function(*args, **kwargs)
>  File "dbrepro.py", line 44, in associate
>    ip = db.fixed_ip_associate_pool(ctxt, 1, instance_id=val)
>  File "/Users/vishvananda/os/nova/nova/db/api.py", line 352, in fixed_ip_associate_pool
>    instance_id, host)
>  File "/Users/vishvananda/os/nova/nova/db/sqlalchemy/api.py", line 102, in wrapper
>    return f(*args, **kwargs)
>  File "/Users/vishvananda/os/nova/nova/db/sqlalchemy/api.py", line 725, in fixed_ip_associate_pool
>    filter_by(host=None).\
>  File "/Library/Python/2.7/site-packages/sqlalchemy/orm/query.py", line 1496, in first
>    ret = list(self[0:1])
>  File "/Library/Python/2.7/site-packages/sqlalchemy/orm/query.py", line 1405, in __getitem__
>    return list(res)
>  File "/Library/Python/2.7/site-packages/sqlalchemy/orm/query.py", line 1669, in instances
>    fetch = cursor.fetchall()
>  File "/Library/Python/2.7/site-packages/sqlalchemy/engine/base.py", line 2383, in fetchall
>    l = self.process_rows(self._fetchall_impl())
>  File "/Library/Python/2.7/site-packages/sqlalchemy/engine/base.py", line 2366, in process_rows
>    keymap = metadata._keymap
> AttributeError: 'NoneType' object has no attribute '_keymap'
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp