yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1441382] Re: IPv6 SLAAC subnet Tempest tests fail due to IntegrityError

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>
Date: Thu, 23 Apr 2015 18:40:18 -0000
Reply-to: Bug 1441382 <1441382@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Changed in: neutron/kilo
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1441382

Title:
  IPv6 SLAAC subnet Tempest tests fail due to IntegrityError

Status in OpenStack Neutron (virtual network service):
  Fix Committed
Status in neutron kilo series:
  Fix Released

Bug description:
  If 2 Tempest tests are run concurrently as follows:
  Test 1: Any test that uses DHCP and cleans up the DHCP port as part of cleanup. For example:
      tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_router
  Test 2: Any test that creates an IPv6 SLAAC subnet
      tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_64_subnets
  and this patch has been applied to eliminate DB access deadlocks:
      https://review.openstack.org/#/c/170690/
  Then occasionally Test 2 will fail in subnet create, and the following error can be observed
  in the neutron server log (q-svc.log):

  TRACE neutron.api.v2.resource DBReferenceError: (IntegrityError)
  (1452, 'Cannot add or update a child row: a foreign key constraint
  fails (`neutron`.`ipallocations`, CONSTRAINT `ipallocations_ibfk_2`
  FOREIGN KEY (`port_id`) REFERENCES `ports` (`id`) ON DELETE CASCADE)')
  'INSERT INTO ipallocations (port_id, ip_address, subnet_id,
  network_id) VALUES (%s, %s, %s, %s)' ('dc359c7e-59b1-46d2-966f-
  194bc7fa0ffb', '2003::f816:3eff:fea6:41d3', 'dc9f1ac8-92c1-4f0e-
  8b86-524824861fa3', '45e4e8ea-81ed-46ad-a753-cdca392adb2a')

  Here is a full traceback:

  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource Traceback (most recent call last):
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/resource.py", line 83, in resource
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     result = method(request=request, **args)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 461, in create
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     obj = obj_creator(request.context, **kwargs)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 794, in create_subnet
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     result, mech_context = self._create_subnet_db(context, subnet)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 785, in _create_subnet_db
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     result = super(Ml2Plugin, self).create_subnet(context, subnet)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1349, in create_subnet
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     raise n_exc.BadRequest(resource='subnets', msg=msg)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1285, in _create_subnet_from_implicit_pool
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     # If this subnet supports auto-addressing, then update any
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1379, in _add_auto_addrs_on_network_ports
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     except db_exc.DBReferenceError:
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 470, in __exit__
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     self.rollback()
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     compat.reraise(exc_type, exc_value, exc_tb)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 467, in __exit__
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     self.commit()
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 377, in commit
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     self._prepare_impl()
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 357, in _prepare_impl
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     self.session.flush()
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1919, in flush
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     self._flush(objects)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2037, in _flush
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     transaction.rollback(_capture_exception=True)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     compat.reraise(exc_type, exc_value, exc_tb)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2001, in _flush
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     flush_context.execute()
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     rec.execute(self)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 526, in execute
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     uow
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 65, in save_obj
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     mapper, table, insert)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 570, in _emit_insert_statements
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     execute(statement, multiparams)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 729, in execute
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     return meth(self, multiparams, params)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     return connection._execute_clauseelement(self, multiparams, params)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     compiled_sql, distilled_params
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     context)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/compat/handle_error.py", line 261, in _handle_dbapi_exception
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     e, statement, parameters, cursor, context)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1156, in _handle_dbapi_exception
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     util.raise_from_cause(newraise, exc_info)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     reraise(type(exception), exception, tb=exc_tb)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     context)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 436, in do_execute
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     cursor.execute(statement, parameters)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     self.errorhandler(self, exc, value)
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource     raise errorclass, errorvalue
  2015-04-07 16:39:16.026 TRACE neutron.api.v2.resource DBReferenceError: (IntegrityError) (1452, 'Cannot add or update a child row: a foreign key constraint fails (`neutron`.`ipallocations`, CONSTRAINT `ipallocations_ibfk_2` FOREIGN KEY (`port_id`) REFERENCES `ports` (`id`) ON DELETE CASCADE)') 'INSERT INTO ipallocations (port_id, ip_address, subnet_id, network_id) VALUES (%s, %s, %s, %s)' ('dc359c7e-59b1-46d2-966f-194bc7fa0ffb', '2003::f816:3eff:fea6:41d3', 'dc9f1ac8-92c1-4f0e-8b86-524824861fa3', '45e4e8ea-81ed-46ad-a753-cdca392adb2a')

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1441382/+subscriptions
References

[Bug 1441382] [NEW] IPv6 SLAAC subnet Tempest tests fail due to IntegrityError
From: Dane LeBlanc, 2015-04-07