← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #10818

[Bug 1288379] [NEW] db Deadlock detected when running 'delete_port

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

I'm seeing Deadlocks when deleting large numbers of VMs in a multinode
system.

The port delete fails, and then ports are left behind after the VMs are deleted.
VMs cannot be created as the IP are not released.
The ports have to be manually deleted. 


2014-02-24 15:18:12.606 1819 ERROR neutron.api.v2.resource [-] delete failed
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource Traceback (most recent call last):
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/api/v2/resource.py", line 84, in resource
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     result = method(request=request, **args)
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py", line 432, in delete
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     obj_deleter(request.context, id, **kwargs)
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/plugins/openvswitch/ovs_neutron_plugin.py", line 634, in delete_port
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     super(OVSNeutronPluginV2, self).delete_port(context, id)
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/db/db_base_plugin_v2.py", line 1403, in delete_port
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     self._delete_port(context, id)
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/db/db_base_plugin_v2.py", line 1425, in _delete_port
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     a['ip_address'])
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/db/db_base_plugin_v2.py", line 415, in _recycle_ip
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     ip_address)
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/db/db_base_plugin_v2.py", line 449, in _delete_ip_allocation
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     subnet_id=subnet_id).delete()
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2581, in delete
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     delete_op.exec_()
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 816, in exec_
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     self._do_exec()
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 942, in _do_exec
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     params=self.query._params)
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/db/sqlalchemy/session.py", line 531, in _wrap
2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     _raise_if_deadlock_error(e, get_engine().name)

** Affects: neutron
     Importance: Undecided
     Assignee: Robert Pothier (rpothier)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Robert Pothier (rpothier)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1288379

Title:
  db Deadlock detected when running 'delete_port

Status in OpenStack Neutron (virtual network service):
  New

Bug description:
  I'm seeing Deadlocks when deleting large numbers of VMs in a multinode
  system.

  The port delete fails, and then ports are left behind after the VMs are deleted.
  VMs cannot be created as the IP are not released.
  The ports have to be manually deleted. 

  
  2014-02-24 15:18:12.606 1819 ERROR neutron.api.v2.resource [-] delete failed
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource Traceback (most recent call last):
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/api/v2/resource.py", line 84, in resource
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     result = method(request=request, **args)
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py", line 432, in delete
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     obj_deleter(request.context, id, **kwargs)
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/plugins/openvswitch/ovs_neutron_plugin.py", line 634, in delete_port
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     super(OVSNeutronPluginV2, self).delete_port(context, id)
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/db/db_base_plugin_v2.py", line 1403, in delete_port
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     self._delete_port(context, id)
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/db/db_base_plugin_v2.py", line 1425, in _delete_port
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     a['ip_address'])
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/db/db_base_plugin_v2.py", line 415, in _recycle_ip
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     ip_address)
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/db/db_base_plugin_v2.py", line 449, in _delete_ip_allocation
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     subnet_id=subnet_id).delete()
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2581, in delete
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     delete_op.exec_()
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 816, in exec_
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     self._do_exec()
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 942, in _do_exec
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     params=self.query._params)
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/db/sqlalchemy/session.py", line 531, in _wrap
  2014-02-24 15:18:12.606 1819 TRACE neutron.api.v2.resource     _raise_if_deadlock_error(e, get_engine().name)

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1288379/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next