yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1623402] [NEW] ipam leaks DBReference error on deleted subnet

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Kevin Benton <1623402@xxxxxxxxxxxxxxxxxx>
Date: Wed, 14 Sep 2016 09:35:36 -0000
Reply-to: Bug 1623402 <1623402@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

spotted in rally run with lots of concurrent subnet operations:


http://logs.openstack.org/11/369511/3/check/gate-rally-dsvm-neutron-rally/b188655/logs/screen-q-svc.txt.gz#_2016-09-14_07_20_10_547


2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource [req-14b41117-5a1b-4b0d-9962-14e9adb7048a c_rally_b76aa7ea_hhhcxI8u -] delete failed: Exception deleting fixed_ip from port 1098dc9c-dd9e-4a11-8e6e-b999218d55aa
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource Traceback (most recent call last):
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/resource.py", line 79, in resource
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     result = method(request=request, **args)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 555, in delete
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return self._delete(request, id, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 88, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 84, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 124, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     traceback.format_exc())
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 119, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 577, in _delete
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     obj_deleter(request.context, id, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/common/utils.py", line 618, in inner
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(self, context, *args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 159, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return method(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 88, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 84, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 124, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     traceback.format_exc())
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 119, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 1169, in delete_subnet
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     "port %s"), port_id)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 1161, in delete_subnet
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.update_port(context, port_id, data)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/common/utils.py", line 618, in inner
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(self, context, *args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 159, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return method(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 88, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 84, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 124, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     traceback.format_exc())
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 119, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 1383, in update_port
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     port)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 159, in wrapped
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return method(*args, **kwargs)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 1221, in update_port
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     new_port=new_port)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/ipam_backend_mixin.py", line 739, in update_port
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     new_port.get('mac_address'))
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/ipam_pluggable_backend.py", line 369, in update_port_with_ips
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     revert_on_fail=False)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/ipam_pluggable_backend.py", line 339, in update_port_with_ips
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     ip['subnet_id'], ip['ip_address'])
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_common.py", line 104, in _delete_ip_allocation
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     subnet_id=subnet_id)):
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2760, in __iter__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.session._autoflush()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1293, in _autoflush
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.flush()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2019, in flush
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self._flush(objects)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2137, in _flush
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     transaction.rollback(_capture_exception=True)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     compat.reraise(exc_type, exc_value, exc_tb)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2101, in _flush
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     flush_context.execute()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 373, in execute
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     rec.execute(self)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 532, in execute
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     uow
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 174, in save_obj
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     mapper, table, insert)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 767, in _emit_insert_statements
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     execute(statement, multiparams)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return meth(self, multiparams, params)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return connection._execute_clauseelement(self, multiparams, params)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     compiled_sql, distilled_params
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     context)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     util.raise_from_cause(newraise, exc_info)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     reraise(type(exception), exception, tb=exc_tb, cause=cause)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     context)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     cursor.execute(statement, parameters)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 167, in execute
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     result = self._query(query)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 323, in _query
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     conn.query(q)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 836, in query
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1020, in _read_query_result
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     result.read()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1303, in read
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     first_packet = self.connection._read_packet()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 982, in _read_packet
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     packet.check_error()
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 394, in check_error
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     err.raise_mysql_exception(self._data)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 120, in raise_mysql_exception
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     _check_mysql_exception(errinfo)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 112, in _check_mysql_exception
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     raise errorclass(errno, errorvalue)
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource DBReferenceError: (pymysql.err.IntegrityError) (1452, u'Cannot add or update a child row: a foreign key constraint fails (`neutron`.`ipamallocations`, CONSTRAINT `ipamallocations_ibfk_1` FOREIGN KEY (`ipam_subnet_id`) REFERENCES `ipamsubnets` (`id`) ON DELETE CASCADE)') [SQL: u'INSERT INTO ipamallocations (ip_address, status, ipam_subnet_id) VALUES (%(ip_address)s, %(status)s, %(ipam_subnet_id)s)'] [parameters: {'status': 'ALLOCATED', 'ip_address': '1.1.0.74', 'ipam_subnet_id': u'1f01b696-8306-43d3-83e0-2642922043b9'}]
2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource

** Affects: neutron
     Importance: Undecided
     Assignee: Kevin Benton (kevinbenton)
         Status: New

** Changed in: neutron
    Milestone: None => newton-rc1

** Changed in: neutron
     Assignee: (unassigned) => Kevin Benton (kevinbenton)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1623402

Title:
  ipam leaks DBReference error on deleted subnet

Status in neutron:
  New

Bug description:
  spotted in rally run with lots of concurrent subnet operations:

  
  http://logs.openstack.org/11/369511/3/check/gate-rally-dsvm-neutron-rally/b188655/logs/screen-q-svc.txt.gz#_2016-09-14_07_20_10_547


  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource [req-14b41117-5a1b-4b0d-9962-14e9adb7048a c_rally_b76aa7ea_hhhcxI8u -] delete failed: Exception deleting fixed_ip from port 1098dc9c-dd9e-4a11-8e6e-b999218d55aa
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource Traceback (most recent call last):
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/resource.py", line 79, in resource
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     result = method(request=request, **args)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 555, in delete
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return self._delete(request, id, **kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 88, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 84, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 124, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     traceback.format_exc())
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 119, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 577, in _delete
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     obj_deleter(request.context, id, **kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/common/utils.py", line 618, in inner
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(self, context, *args, **kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 159, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return method(*args, **kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 88, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 84, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 124, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     traceback.format_exc())
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 119, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 1169, in delete_subnet
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     "port %s"), port_id)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 1161, in delete_subnet
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.update_port(context, port_id, data)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/common/utils.py", line 618, in inner
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(self, context, *args, **kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 159, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return method(*args, **kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 88, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 84, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 124, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     traceback.format_exc())
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 119, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 1383, in update_port
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     port)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/api.py", line 159, in wrapped
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return method(*args, **kwargs)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 1221, in update_port
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     new_port=new_port)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/ipam_backend_mixin.py", line 739, in update_port
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     new_port.get('mac_address'))
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/ipam_pluggable_backend.py", line 369, in update_port_with_ips
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     revert_on_fail=False)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/ipam_pluggable_backend.py", line 339, in update_port_with_ips
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     ip['subnet_id'], ip['ip_address'])
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_common.py", line 104, in _delete_ip_allocation
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     subnet_id=subnet_id)):
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2760, in __iter__
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.session._autoflush()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1293, in _autoflush
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self.flush()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2019, in flush
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self._flush(objects)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2137, in _flush
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     transaction.rollback(_capture_exception=True)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     compat.reraise(exc_type, exc_value, exc_tb)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2101, in _flush
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     flush_context.execute()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 373, in execute
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     rec.execute(self)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 532, in execute
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     uow
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 174, in save_obj
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     mapper, table, insert)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 767, in _emit_insert_statements
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     execute(statement, multiparams)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return meth(self, multiparams, params)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     return connection._execute_clauseelement(self, multiparams, params)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     compiled_sql, distilled_params
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     context)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     util.raise_from_cause(newraise, exc_info)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     reraise(type(exception), exception, tb=exc_tb, cause=cause)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     context)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     cursor.execute(statement, parameters)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 167, in execute
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     result = self._query(query)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 323, in _query
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     conn.query(q)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 836, in query
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1020, in _read_query_result
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     result.read()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1303, in read
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     first_packet = self.connection._read_packet()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 982, in _read_packet
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     packet.check_error()
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 394, in check_error
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     err.raise_mysql_exception(self._data)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 120, in raise_mysql_exception
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     _check_mysql_exception(errinfo)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 112, in _check_mysql_exception
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource     raise errorclass(errno, errorvalue)
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource DBReferenceError: (pymysql.err.IntegrityError) (1452, u'Cannot add or update a child row: a foreign key constraint fails (`neutron`.`ipamallocations`, CONSTRAINT `ipamallocations_ibfk_1` FOREIGN KEY (`ipam_subnet_id`) REFERENCES `ipamsubnets` (`id`) ON DELETE CASCADE)') [SQL: u'INSERT INTO ipamallocations (ip_address, status, ipam_subnet_id) VALUES (%(ip_address)s, %(status)s, %(ipam_subnet_id)s)'] [parameters: {'status': 'ALLOCATED', 'ip_address': '1.1.0.74', 'ipam_subnet_id': u'1f01b696-8306-43d3-83e0-2642922043b9'}]
  2016-09-14 07:20:10.547 28514 ERROR neutron.api.v2.resource

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1623402/+subscriptions
Follow ups

[Bug 1623402] Re: ipam leaks DBReference error on deleted subnet
From: OpenStack Infra, 2016-09-16