yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1262162] [NEW] nvp: IP allocation conflict while deleting metadata network

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Salvatore Orlando <1262162@xxxxxxxxxxxxxxxxxx>
Date: Wed, 18 Dec 2013 11:18:45 -0000
Reply-to: Bug 1262162 <1262162@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

This has been found while running automated testing:

71441:2013-12-17 08:33:57.127 DEBUG neutron.db.db_base_plugin_v2 [req-ba712d02-26ac-4e31-8613-de1a4558b028 None None] Recycle 169.254.169.254 from (pid=29594) _recycle_ip /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:366
71442:2013-12-17 08:33:57.130 DEBUG neutron.db.db_base_plugin_v2 [req-ba712d02-26ac-4e31-8613-de1a4558b028 None None] Recycle: created new 169.254.169.254-169.254.169.254 from (pid=29594) _recycle_ip /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:413
71443:2013-12-17 08:33:57.130 DEBUG neutron.db.db_base_plugin_v2 [req-ba712d02-26ac-4e31-8613-de1a4558b028 None None] Delete allocated IP 169.254.169.254 (2000591b-eb4a-4056-9055-0e961500b5b8/459fc4a2-ba66-4fb1-89d8-badd0b6c65c7) from (pid=29594) _delete_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:444
71444:2013-12-17 08:33:57.132 ERROR neutron.openstack.common.db.sqlalchemy.session [req-ba712d02-26ac-4e31-8613-de1a4558b028 None None] DB exception wrapped.
71445:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session Traceback (most recent call last):
71446:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/opt/stack/neutron/neutron/openstack/common/db/sqlalchemy/session.py", line 524, in _wrap
71447:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     return f(*args, **kwargs)
71448:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/opt/stack/neutron/neutron/openstack/common/db/sqlalchemy/session.py", line 718, in flush
71449:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     return super(Session, self).flush(*args, **kwargs)
71450:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1734, in flush
71451:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     self._flush(objects)
71452:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1805, in _flush
71453:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     flush_context.execute()
71454:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 331, in execute
71455:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     rec.execute(self)
71456:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 475, in execute
71457:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     uow
71458:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 45, in save_obj
71459:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     uowtransaction)
71460:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 165, in _organize_states_for_save
71461:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     state_str(existing)))
71462:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session FlushError: New instance <IPAvailabilityRange at 0x43874d0> with identity key (<class 'neutron.db.models_v2.IPAvailabilityRange'>, (u'4be61a8f-0928-40bb-b736-6855b8c94e74', u'169.254.169.254', u'169.254.169.254')) conflicts with persistent instance <IPAvailabilityRange at 0x458a890>
71463:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session 
71464:2013-12-17 08:33:57.137 DEBUG neutron.plugins.nicira.api_client.request [-] [3453] Completed request 'PUT https://192.168.1.8:443//ws.v1/lswitch/2000591b-eb4a-4056-9055-0e961500b5b8/lport/ed58a6cc-3874-4ef9-85d4-a529f3c09cdb/attachment': 200 (0.08 seconds) from (pid=29594) _issue_request /opt/stack/neutron/neutron/plugins/nicira/api_client/request.py:141
71465:2013-12-17 08:33:57.137 DEBUG neutron.plugins.nicira.api_client.request [-] Reading X-Nvp-config-Generation response header: '96045' from (pid=29594) _issue_request /opt/stack/neutron/neutron/plugins/nicira/api_client/request.py:146
71466:2013-12-17 08:33:57.137 DEBUG neutron.plugins.nicira.api_client.client [-] [3453] Released connection https://192.168.1.8:443. 10 connection(s) available. from (pid=29594) release_connection /opt/stack/neutron/neutron/plugins/nicira/api_client/client.py:189
71467:2013-12-17 08:33:57.138 ERROR neutron.api.v2.resource [req-ba712d02-26ac-4e31-8613-de1a4558b028 None None] remove_router_interface failed
71468:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource Traceback (most recent call last):
71469:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/resource.py", line 84, in resource
71470:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     result = method(request=request, **args)
71471:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 191, in _handle_action
71472:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     return getattr(self._plugin, name)(*arg_list, **kwargs)
71473:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/nicira/NeutronPlugin.py", line 1703, in remove_router_interface
71474:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     context, router_id, interface=info)
71475:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/nicira/dhcpmeta_modes.py", line 116, in handle_router_metadata_access
71476:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     router_id, interface)
71477:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/nicira/dhcp_meta/rpc.py", line 138, in handle_router_metadata_access
71478:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     plugin, ctx_elevated, router_id, ports)
71479:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/nicira/dhcp_meta/rpc.py", line 221, in _destroy_metadata_access_network
71480:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     plugin.delete_network(context, meta_net_id)
71481:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/nicira/NeutronPlugin.py", line 1015, in delete_network
71482:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     super(NvpPluginV2, self).delete_network(context, id)
71483:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1006, in delete_network
71484:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     self._delete_port(context, port['id'])
71485:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1439, in _delete_port
71486:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     a['ip_address'])
71487:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 415, in _recycle_ip
71488:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     ip_address)
71489:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 448, in _delete_ip_allocation
71490:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     subnet_id=subnet_id).delete()
71491:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2642, in delete
71492:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     session._autoflush()
71493:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1127, in _autoflush
71494:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     self.flush()
71495:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/openstack/common/db/sqlalchemy/session.py", line 545, in _wrap
71496:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     raise exception.DBError(e)
71497:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource DBError: New instance <IPAvailabilityRange at 0x43874d0> with identity key (<class 'neutron.db.models_v2.IPAvailabilityRange'>, (u'4be61a8f-0928-40bb-b736-6855b8c94e74', u'169.254.169.254', u'169.254.169.254')) conflicts with persistent instance <IPAvailabilityRange at 0x458a890>
71498:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource 

Tempest failed with "500 Internal Server Error"
Root cause is uncertain.

** Affects: neutron
     Importance: Undecided
     Assignee: Salvatore Orlando (salvatore-orlando)
         Status: New


** Tags: nicira

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1262162

Title:
  nvp: IP allocation conflict while deleting metadata network

Status in OpenStack Neutron (virtual network service):
  New

Bug description:
  This has been found while running automated testing:

  71441:2013-12-17 08:33:57.127 DEBUG neutron.db.db_base_plugin_v2 [req-ba712d02-26ac-4e31-8613-de1a4558b028 None None] Recycle 169.254.169.254 from (pid=29594) _recycle_ip /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:366
  71442:2013-12-17 08:33:57.130 DEBUG neutron.db.db_base_plugin_v2 [req-ba712d02-26ac-4e31-8613-de1a4558b028 None None] Recycle: created new 169.254.169.254-169.254.169.254 from (pid=29594) _recycle_ip /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:413
  71443:2013-12-17 08:33:57.130 DEBUG neutron.db.db_base_plugin_v2 [req-ba712d02-26ac-4e31-8613-de1a4558b028 None None] Delete allocated IP 169.254.169.254 (2000591b-eb4a-4056-9055-0e961500b5b8/459fc4a2-ba66-4fb1-89d8-badd0b6c65c7) from (pid=29594) _delete_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:444
  71444:2013-12-17 08:33:57.132 ERROR neutron.openstack.common.db.sqlalchemy.session [req-ba712d02-26ac-4e31-8613-de1a4558b028 None None] DB exception wrapped.
  71445:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session Traceback (most recent call last):
  71446:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/opt/stack/neutron/neutron/openstack/common/db/sqlalchemy/session.py", line 524, in _wrap
  71447:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     return f(*args, **kwargs)
  71448:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/opt/stack/neutron/neutron/openstack/common/db/sqlalchemy/session.py", line 718, in flush
  71449:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     return super(Session, self).flush(*args, **kwargs)
  71450:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1734, in flush
  71451:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     self._flush(objects)
  71452:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1805, in _flush
  71453:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     flush_context.execute()
  71454:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 331, in execute
  71455:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     rec.execute(self)
  71456:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 475, in execute
  71457:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     uow
  71458:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 45, in save_obj
  71459:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     uowtransaction)
  71460:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 165, in _organize_states_for_save
  71461:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session     state_str(existing)))
  71462:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session FlushError: New instance <IPAvailabilityRange at 0x43874d0> with identity key (<class 'neutron.db.models_v2.IPAvailabilityRange'>, (u'4be61a8f-0928-40bb-b736-6855b8c94e74', u'169.254.169.254', u'169.254.169.254')) conflicts with persistent instance <IPAvailabilityRange at 0x458a890>
  71463:2013-12-17 08:33:57.132 TRACE neutron.openstack.common.db.sqlalchemy.session 
  71464:2013-12-17 08:33:57.137 DEBUG neutron.plugins.nicira.api_client.request [-] [3453] Completed request 'PUT https://192.168.1.8:443//ws.v1/lswitch/2000591b-eb4a-4056-9055-0e961500b5b8/lport/ed58a6cc-3874-4ef9-85d4-a529f3c09cdb/attachment': 200 (0.08 seconds) from (pid=29594) _issue_request /opt/stack/neutron/neutron/plugins/nicira/api_client/request.py:141
  71465:2013-12-17 08:33:57.137 DEBUG neutron.plugins.nicira.api_client.request [-] Reading X-Nvp-config-Generation response header: '96045' from (pid=29594) _issue_request /opt/stack/neutron/neutron/plugins/nicira/api_client/request.py:146
  71466:2013-12-17 08:33:57.137 DEBUG neutron.plugins.nicira.api_client.client [-] [3453] Released connection https://192.168.1.8:443. 10 connection(s) available. from (pid=29594) release_connection /opt/stack/neutron/neutron/plugins/nicira/api_client/client.py:189
  71467:2013-12-17 08:33:57.138 ERROR neutron.api.v2.resource [req-ba712d02-26ac-4e31-8613-de1a4558b028 None None] remove_router_interface failed
  71468:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource Traceback (most recent call last):
  71469:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/resource.py", line 84, in resource
  71470:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     result = method(request=request, **args)
  71471:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 191, in _handle_action
  71472:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     return getattr(self._plugin, name)(*arg_list, **kwargs)
  71473:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/nicira/NeutronPlugin.py", line 1703, in remove_router_interface
  71474:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     context, router_id, interface=info)
  71475:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/nicira/dhcpmeta_modes.py", line 116, in handle_router_metadata_access
  71476:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     router_id, interface)
  71477:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/nicira/dhcp_meta/rpc.py", line 138, in handle_router_metadata_access
  71478:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     plugin, ctx_elevated, router_id, ports)
  71479:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/nicira/dhcp_meta/rpc.py", line 221, in _destroy_metadata_access_network
  71480:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     plugin.delete_network(context, meta_net_id)
  71481:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/nicira/NeutronPlugin.py", line 1015, in delete_network
  71482:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     super(NvpPluginV2, self).delete_network(context, id)
  71483:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1006, in delete_network
  71484:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     self._delete_port(context, port['id'])
  71485:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1439, in _delete_port
  71486:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     a['ip_address'])
  71487:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 415, in _recycle_ip
  71488:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     ip_address)
  71489:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 448, in _delete_ip_allocation
  71490:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     subnet_id=subnet_id).delete()
  71491:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2642, in delete
  71492:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     session._autoflush()
  71493:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1127, in _autoflush
  71494:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     self.flush()
  71495:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/openstack/common/db/sqlalchemy/session.py", line 545, in _wrap
  71496:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource     raise exception.DBError(e)
  71497:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource DBError: New instance <IPAvailabilityRange at 0x43874d0> with identity key (<class 'neutron.db.models_v2.IPAvailabilityRange'>, (u'4be61a8f-0928-40bb-b736-6855b8c94e74', u'169.254.169.254', u'169.254.169.254')) conflicts with persistent instance <IPAvailabilityRange at 0x458a890>
  71498:2013-12-17 08:33:57.138 TRACE neutron.api.v2.resource 

  Tempest failed with "500 Internal Server Error"
  Root cause is uncertain.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1262162/+subscriptions
Follow ups

[Bug 1262162] [NEW] nvp: IP allocation conflict while deleting metadata network
From: Salvatore Orlando, 2013-12-18
References

[Bug 1262162] [NEW] nvp: IP allocation conflict while deleting metadata network
From: Salvatore Orlando, 2013-12-18