yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1332923] [NEW] Deadlock updating port with fixed ips

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Eugene Nikanorov <enikanorov@xxxxxxxxxxxx>
Date: Sun, 22 Jun 2014 08:13:33 -0000
Reply-to: Bug 1332923 <1332923@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Traceback:

 TRACE neutron.api.v2.resource Traceback (most recent call last):
 TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/resource.py", line 87, in resource
 TRACE neutron.api.v2.resource     result = method(request=request, **args)
 TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 531, in update
 TRACE neutron.api.v2.resource     obj = obj_updater(request.context, id, **kwargs)
 TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 682, in update_port
 TRACE neutron.api.v2.resource     port)
 TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 1497, in update_port
 TRACE neutron.api.v2.resource     p['fixed_ips'])
 TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 650, in _update_ips_for_port
 TRACE neutron.api.v2.resource     ips = self._allocate_fixed_ips(context, network, to_add)
 TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 612, in _allocate_fixed_ips
 TRACE neutron.api.v2.resource     result = self._generate_ip(context, subnets)
 TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 364, in _generate_ip
 TRACE neutron.api.v2.resource     return NeutronDbPluginV2._try_generate_ip(context, subnets)
 TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 381, in _try_generate_ip
 TRACE neutron.api.v2.resource     range = range_qry.filter_by(subnet_id=subnet['id']).first()
 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2333, in first
 TRACE neutron.api.v2.resource     ret = list(self[0:1])
 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2200, in __getitem__
 TRACE neutron.api.v2.resource     return list(res)
 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2404, in __iter__
 TRACE neutron.api.v2.resource     return self._execute_and_instances(context)
 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2419, in _execute_and_instances
 TRACE neutron.api.v2.resource     result = conn.execute(querycontext.statement, self._params)
 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 720, in execute
 TRACE neutron.api.v2.resource     return meth(self, multiparams, params)
 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 317, in _execute_on_connection
 TRACE neutron.api.v2.resource     return connection._execute_clauseelement(self, multiparams, params)
 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 817, in _execute_clauseelement
 TRACE neutron.api.v2.resource     compiled_sql, distilled_params
 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 947, in _execute_context
 TRACE neutron.api.v2.resource     context)
 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1108, in _handle_dbapi_exception
 TRACE neutron.api.v2.resource     exc_info
 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 185, in raise_from_cause
 TRACE neutron.api.v2.resource     reraise(type(exception), exception, tb=exc_tb)
 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 940, in _execute_context
 TRACE neutron.api.v2.resource     context)
 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 435, in do_execute
 TRACE neutron.api.v2.resource     cursor.execute(statement, parameters)
 TRACE neutron.api.v2.resource DBAPIError: (TransactionRollbackError) deadlock detected
 TRACE neutron.api.v2.resource DETAIL:  Process 21690 waits for ShareLock on transaction 10397; blocked by process 21692.
 TRACE neutron.api.v2.resource Process 21692 waits for ShareLock on transaction 10396; blocked by process 21690.
 TRACE neutron.api.v2.resource HINT:  See server log for query details.
 TRACE neutron.api.v2.resource  'SELECT ipavailabilityranges.allocation_pool_id AS ipavailabilityranges_allocation_pool_id, ipavailabilityranges.first_ip AS ipavailabilityranges_first_ip, ipavailabilityranges.last_ip AS ipavailabilityranges_last_ip \nFROM ipavailabilityranges JOIN ipallocationpools ON ipallocationpools.id = ipavailabilityranges.allocation_pool_id \nWHERE ipallocationpools.subnet_id = %(subnet_id_1)s \n LIMIT %(param_1)s FOR UPDATE' {'param_1': 1, 'subnet_id_1': u'7dd0bab8-e96c-4a23-8603-c538239d112d'}

http://logs.openstack.org/38/95138/8/check/check-tempest-dsvm-neutron-
pg/99dc5cb/logs/screen-q-svc.txt.gz?level=TRACE#_2014-06-21_18_07_30_852

** Affects: neutron
     Importance: High
         Status: Confirmed


** Tags: db gate-failure ml2

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1332923

Title:
  Deadlock updating port with fixed ips

Status in OpenStack Neutron (virtual network service):
  Confirmed

Bug description:
  Traceback:

   TRACE neutron.api.v2.resource Traceback (most recent call last):
   TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/resource.py", line 87, in resource
   TRACE neutron.api.v2.resource     result = method(request=request, **args)
   TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 531, in update
   TRACE neutron.api.v2.resource     obj = obj_updater(request.context, id, **kwargs)
   TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 682, in update_port
   TRACE neutron.api.v2.resource     port)
   TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 1497, in update_port
   TRACE neutron.api.v2.resource     p['fixed_ips'])
   TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 650, in _update_ips_for_port
   TRACE neutron.api.v2.resource     ips = self._allocate_fixed_ips(context, network, to_add)
   TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 612, in _allocate_fixed_ips
   TRACE neutron.api.v2.resource     result = self._generate_ip(context, subnets)
   TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 364, in _generate_ip
   TRACE neutron.api.v2.resource     return NeutronDbPluginV2._try_generate_ip(context, subnets)
   TRACE neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 381, in _try_generate_ip
   TRACE neutron.api.v2.resource     range = range_qry.filter_by(subnet_id=subnet['id']).first()
   TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2333, in first
   TRACE neutron.api.v2.resource     ret = list(self[0:1])
   TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2200, in __getitem__
   TRACE neutron.api.v2.resource     return list(res)
   TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2404, in __iter__
   TRACE neutron.api.v2.resource     return self._execute_and_instances(context)
   TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2419, in _execute_and_instances
   TRACE neutron.api.v2.resource     result = conn.execute(querycontext.statement, self._params)
   TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 720, in execute
   TRACE neutron.api.v2.resource     return meth(self, multiparams, params)
   TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 317, in _execute_on_connection
   TRACE neutron.api.v2.resource     return connection._execute_clauseelement(self, multiparams, params)
   TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 817, in _execute_clauseelement
   TRACE neutron.api.v2.resource     compiled_sql, distilled_params
   TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 947, in _execute_context
   TRACE neutron.api.v2.resource     context)
   TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1108, in _handle_dbapi_exception
   TRACE neutron.api.v2.resource     exc_info
   TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 185, in raise_from_cause
   TRACE neutron.api.v2.resource     reraise(type(exception), exception, tb=exc_tb)
   TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 940, in _execute_context
   TRACE neutron.api.v2.resource     context)
   TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 435, in do_execute
   TRACE neutron.api.v2.resource     cursor.execute(statement, parameters)
   TRACE neutron.api.v2.resource DBAPIError: (TransactionRollbackError) deadlock detected
   TRACE neutron.api.v2.resource DETAIL:  Process 21690 waits for ShareLock on transaction 10397; blocked by process 21692.
   TRACE neutron.api.v2.resource Process 21692 waits for ShareLock on transaction 10396; blocked by process 21690.
   TRACE neutron.api.v2.resource HINT:  See server log for query details.
   TRACE neutron.api.v2.resource  'SELECT ipavailabilityranges.allocation_pool_id AS ipavailabilityranges_allocation_pool_id, ipavailabilityranges.first_ip AS ipavailabilityranges_first_ip, ipavailabilityranges.last_ip AS ipavailabilityranges_last_ip \nFROM ipavailabilityranges JOIN ipallocationpools ON ipallocationpools.id = ipavailabilityranges.allocation_pool_id \nWHERE ipallocationpools.subnet_id = %(subnet_id_1)s \n LIMIT %(param_1)s FOR UPDATE' {'param_1': 1, 'subnet_id_1': u'7dd0bab8-e96c-4a23-8603-c538239d112d'}

  http://logs.openstack.org/38/95138/8/check/check-tempest-dsvm-neutron-
  pg/99dc5cb/logs/screen-q-svc.txt.gz?level=TRACE#_2014-06-21_18_07_30_852

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1332923/+subscriptions
Follow ups

[Bug 1332923] Re: Deadlock updating port with fixed ips
From: Rossella Sblendido, 2015-10-22
[Bug 1332923] [NEW] Deadlock updating port with fixed ips
From: Eugene Nikanorov, 2014-06-22
References

[Bug 1332923] [NEW] Deadlock updating port with fixed ips
From: Eugene Nikanorov, 2014-06-22