← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1429015] Re: vmware metadata rollback unit test spurious failure

 

[Expired for neutron because there has been no activity for 60 days.]

** Changed in: neutron
       Status: Incomplete => Expired

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1429015

Title:
  vmware metadata rollback unit test spurious failure

Status in OpenStack Neutron (virtual network service):
  Expired
Status in VMware NSX:
  Expired

Bug description:
  This failure appears under some timing conditions.

  FAIL: neutron.tests.unit.vmware.test_nsx_plugin.TestL3NatTestCase.test_metadata_network_remove_rollback_on_failure
  tags: worker-0
  ----------------------------------------------------------------------
  Traceback (most recent call last):
  _StringException: Empty attachments:
    pythonlogging:'neutron.api.extensions'
    stderr
    stdout

  pythonlogging:'': {{{
  2015-03-05 19:56:15,778     INFO [neutron.manager] Loading core plugin: neutron.plugins.vmware.plugins.base.NsxPluginV2
  2015-03-05 19:56:15,783     INFO [neutron.manager] Service L3_ROUTER_NAT is supported by the core plugin
  2015-03-05 19:56:15,815  WARNING [neutron.notifiers.nova] Authenticating to nova using nova_admin_* options is deprecated. This should be done using an auth plugin, like password
  2015-03-05 19:56:15,817  WARNING [neutron.notifiers.nova] Authenticating to nova using nova_admin_* options is deprecated. This should be done using an auth plugin, like password
  2015-03-05 19:56:15,818  WARNING [neutron.notifiers.nova] Authenticating to nova using nova_admin_* options is deprecated. This should be done using an auth plugin, like password
  2015-03-05 19:56:15,823     INFO [neutron.common.config] Config paste file: /home/administrator/code/neutron/neutron/tests/etc/api-paste.ini.test
  2015-03-05 19:56:15,824  WARNING [neutron.quota] router is already registered.
  2015-03-05 19:56:15,824  WARNING [neutron.notifiers.nova] Authenticating to nova using nova_admin_* options is deprecated. This should be done using an auth plugin, like password
  2015-03-05 19:56:15,825  WARNING [neutron.quota] floatingip is already registered.
  2015-03-05 19:56:15,826  WARNING [neutron.notifiers.nova] Authenticating to nova using nova_admin_* options is deprecated. This should be done using an auth plugin, like password
  2015-03-05 19:56:15,836  WARNING [neutron.openstack.common.policy] Can not find policy directory: policy.d
  2015-03-05 19:56:16,102     INFO [neutron.plugins.vmware.nsxlib.router] No SNAT rules cannot be applied as they are not available in this version of the NSX platform
  2015-03-05 19:56:16,154  WARNING [neutron.scheduler.dhcp_agent_scheduler] No more DHCP agents
  2015-03-05 19:56:16,346     INFO [neutron.plugins.vmware.nsxlib.router] No SNAT rules cannot be applied as they are not available in this version of the NSX platform
  2015-03-05 19:56:16,412    ERROR [neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api] Will not send event subnet_create_end for network efa233eb-b598-445c-a2f1-b9fb50c33030: no agent available. Payload: {'subnet': {'ipv6_ra_mode': None, 'allocation_pools': [{'start': '169.254.169.254', 'end': '169.254.169.254'}], 'host_routes': [], 'ipv6_address_mode': None, 'cidr': '169.254.169.252/30', 'id': '9d4dcfe6-f74b-4c96-b492-7ed091660b25', 'name': u'meta-794a47e9-6cf2-4770-b4c7-2cf46a02d7d9', 'enable_dhcp': True, 'network_id': 'efa233eb-b598-445c-a2f1-b9fb50c33030', 'tenant_id': '', 'dns_nameservers': [], 'gateway_ip': '169.254.169.253', 'ip_version': 4, 'shared': False}}
  2015-03-05 19:56:16,760     INFO [neutron.plugins.vmware.nsxlib.switch] get_port() 4ae1f982-cd4c-4ee9-9712-1290b8139a63 0b1c5a28-c058-4ac1-845d-32b8362edceb
  2015-03-05 19:56:16,812    ERROR [neutron.plugins.vmware.dhcp_meta.rpc] An error occurred while operating on the metadata access network for router:'794a47e9-6cf2-4770-b4c7-2cf46a02d7d9'
  Traceback (most recent call last):
    File "neutron/plugins/vmware/dhcp_meta/rpc.py", line 114, in handle_router_metadata_access
      plugin, ctx_elevated, router_id, ports)
    File "neutron/plugins/vmware/dhcp_meta/rpc.py", line 194, in _destroy_metadata_access_network
      context, router_id, {'port_id': meta_port['id']})
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/mock.py", line 955, in __call__
      return _mock_self._mock_call(*args, **kwargs)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/mock.py", line 1018, in _mock_call
      ret_val = effect(*args, **kwargs)
    File "neutron/tests/unit/vmware/test_nsx_plugin.py", line 935, in side_effect
      raise api_exc.NsxApiException()
  NsxApiException: An unknown exception occurred.
  2015-03-05 19:56:17,053     INFO [neutron.plugins.vmware.nsxlib.switch] get_port() 70141946-f497-4281-91ed-4c5d24c185bd 9f12fc89-7d67-4014-b939-e9c82643559e
  2015-03-05 19:56:17,198    ERROR [neutron.api.v2.resource] delete failed
  Traceback (most recent call last):
    File "neutron/api/v2/resource.py", line 83, in resource
      result = method(request=request, **args)
    File "neutron/api/v2/base.py", line 480, in delete
      obj_deleter(request.context, id, **kwargs)
    File "neutron/plugins/vmware/plugins/base.py", line 1583, in delete_router
      context, router_id, interface=None)
    File "neutron/plugins/vmware/dhcpmeta_modes.py", line 167, in handle_router_metadata_access
      router_id, interface)
    File "neutron/plugins/vmware/dhcp_meta/rpc.py", line 114, in handle_router_metadata_access
      plugin, ctx_elevated, router_id, ports)
    File "neutron/plugins/vmware/dhcp_meta/rpc.py", line 205, in _destroy_metadata_access_network
      {'subnet_id': meta_sub_id})
    File "neutron/plugins/vmware/plugins/base.py", line 1665, in add_router_interface
      context, router_id, interface_info)
    File "neutron/db/l3_dvr_db.py", line 289, in add_router_interface
      router = self._get_router(context, router_id)
    File "neutron/db/l3_db.py", line 122, in _get_router
      router = self._get_by_id(context, Router, router_id)
    File "neutron/db/common_db_mixin.py", line 130, in _get_by_id
      return query.filter(model.id == id).one()
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2395, in one
      ret = list(self)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2438, in __iter__
      return self._execute_and_instances(context)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2451, in _execute_and_instances
      close_with_result=True)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2442, in _connection_from_session
      **kw)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 854, in connection
      close_with_result=close_with_result)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 858, in _connection_for_bind
      return self.transaction._connection_for_bind(engine)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 305, in _connection_for_bind
      self._assert_active()
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 218, in _assert_active
      "This Session's transaction has been rolled back "
  InvalidRequestError: This Session's transaction has been rolled back by a nested rollback() call.  To begin a new transaction, issue Session.rollback() first.
  }}}

  Traceback (most recent call last):
    File "neutron/tests/unit/vmware/test_nsx_plugin.py", line 946, in test_metadata_network_remove_rollback_on_failure
      webob.exc.HTTPOk.code)
    File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
      self.gen.next()
    File "neutron/tests/unit/test_l3_plugin.py", line 415, in router
      self._delete('routers', router['router']['id'])
    File "neutron/tests/unit/test_db_plugin.py", line 471, in _delete
      self.assertEqual(res.status_int, expected_code)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 348, in assertEqual
      self.assertThat(observed, matcher, message)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 433, in assertThat
      raise mismatch_error
  MismatchError: 500 != 204

  Traceback (most recent call last):
  _StringException: Empty attachments:
    pythonlogging:'neutron.api.extensions'
    stderr
    stdout

  pythonlogging:'': {{{
  2015-03-05 19:56:15,778     INFO [neutron.manager] Loading core plugin: neutron.plugins.vmware.plugins.base.NsxPluginV2
  2015-03-05 19:56:15,783     INFO [neutron.manager] Service L3_ROUTER_NAT is supported by the core plugin
  2015-03-05 19:56:15,815  WARNING [neutron.notifiers.nova] Authenticating to nova using nova_admin_* options is deprecated. This should be done using an auth plugin, like password
  2015-03-05 19:56:15,817  WARNING [neutron.notifiers.nova] Authenticating to nova using nova_admin_* options is deprecated. This should be done using an auth plugin, like password
  2015-03-05 19:56:15,818  WARNING [neutron.notifiers.nova] Authenticating to nova using nova_admin_* options is deprecated. This should be done using an auth plugin, like password
  2015-03-05 19:56:15,823     INFO [neutron.common.config] Config paste file: /home/administrator/code/neutron/neutron/tests/etc/api-paste.ini.test
  2015-03-05 19:56:15,824  WARNING [neutron.quota] router is already registered.
  2015-03-05 19:56:15,824  WARNING [neutron.notifiers.nova] Authenticating to nova using nova_admin_* options is deprecated. This should be done using an auth plugin, like password
  2015-03-05 19:56:15,825  WARNING [neutron.quota] floatingip is already registered.
  2015-03-05 19:56:15,826  WARNING [neutron.notifiers.nova] Authenticating to nova using nova_admin_* options is deprecated. This should be done using an auth plugin, like password
  2015-03-05 19:56:15,836  WARNING [neutron.openstack.common.policy] Can not find policy directory: policy.d
  2015-03-05 19:56:16,102     INFO [neutron.plugins.vmware.nsxlib.router] No SNAT rules cannot be applied as they are not available in this version of the NSX platform
  2015-03-05 19:56:16,154  WARNING [neutron.scheduler.dhcp_agent_scheduler] No more DHCP agents
  2015-03-05 19:56:16,346     INFO [neutron.plugins.vmware.nsxlib.router] No SNAT rules cannot be applied as they are not available in this version of the NSX platform
  2015-03-05 19:56:16,412    ERROR [neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api] Will not send event subnet_create_end for network efa233eb-b598-445c-a2f1-b9fb50c33030: no agent available. Payload: {'subnet': {'ipv6_ra_mode': None, 'allocation_pools': [{'start': '169.254.169.254', 'end': '169.254.169.254'}], 'host_routes': [], 'ipv6_address_mode': None, 'cidr': '169.254.169.252/30', 'id': '9d4dcfe6-f74b-4c96-b492-7ed091660b25', 'name': u'meta-794a47e9-6cf2-4770-b4c7-2cf46a02d7d9', 'enable_dhcp': True, 'network_id': 'efa233eb-b598-445c-a2f1-b9fb50c33030', 'tenant_id': '', 'dns_nameservers': [], 'gateway_ip': '169.254.169.253', 'ip_version': 4, 'shared': False}}
  2015-03-05 19:56:16,760     INFO [neutron.plugins.vmware.nsxlib.switch] get_port() 4ae1f982-cd4c-4ee9-9712-1290b8139a63 0b1c5a28-c058-4ac1-845d-32b8362edceb
  2015-03-05 19:56:16,812    ERROR [neutron.plugins.vmware.dhcp_meta.rpc] An error occurred while operating on the metadata access network for router:'794a47e9-6cf2-4770-b4c7-2cf46a02d7d9'
  Traceback (most recent call last):
    File "neutron/plugins/vmware/dhcp_meta/rpc.py", line 114, in handle_router_metadata_access
      plugin, ctx_elevated, router_id, ports)
    File "neutron/plugins/vmware/dhcp_meta/rpc.py", line 194, in _destroy_metadata_access_network
      context, router_id, {'port_id': meta_port['id']})
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/mock.py", line 955, in __call__
      return _mock_self._mock_call(*args, **kwargs)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/mock.py", line 1018, in _mock_call
      ret_val = effect(*args, **kwargs)
    File "neutron/tests/unit/vmware/test_nsx_plugin.py", line 935, in side_effect
      raise api_exc.NsxApiException()
  NsxApiException: An unknown exception occurred.
  2015-03-05 19:56:17,053     INFO [neutron.plugins.vmware.nsxlib.switch] get_port() 70141946-f497-4281-91ed-4c5d24c185bd 9f12fc89-7d67-4014-b939-e9c82643559e
  2015-03-05 19:56:17,198    ERROR [neutron.api.v2.resource] delete failed
  Traceback (most recent call last):
    File "neutron/api/v2/resource.py", line 83, in resource
      result = method(request=request, **args)
    File "neutron/api/v2/base.py", line 480, in delete
      obj_deleter(request.context, id, **kwargs)
    File "neutron/plugins/vmware/plugins/base.py", line 1583, in delete_router
      context, router_id, interface=None)
    File "neutron/plugins/vmware/dhcpmeta_modes.py", line 167, in handle_router_metadata_access
      router_id, interface)
    File "neutron/plugins/vmware/dhcp_meta/rpc.py", line 114, in handle_router_metadata_access
      plugin, ctx_elevated, router_id, ports)
    File "neutron/plugins/vmware/dhcp_meta/rpc.py", line 205, in _destroy_metadata_access_network
      {'subnet_id': meta_sub_id})
    File "neutron/plugins/vmware/plugins/base.py", line 1665, in add_router_interface
      context, router_id, interface_info)
    File "neutron/db/l3_dvr_db.py", line 289, in add_router_interface
      router = self._get_router(context, router_id)
    File "neutron/db/l3_db.py", line 122, in _get_router
      router = self._get_by_id(context, Router, router_id)
    File "neutron/db/common_db_mixin.py", line 130, in _get_by_id
      return query.filter(model.id == id).one()
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2395, in one
      ret = list(self)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2438, in __iter__
      return self._execute_and_instances(context)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2451, in _execute_and_instances
      close_with_result=True)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2442, in _connection_from_session
      **kw)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 854, in connection
      close_with_result=close_with_result)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 858, in _connection_for_bind
      return self.transaction._connection_for_bind(engine)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 305, in _connection_for_bind
      self._assert_active()
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 218, in _assert_active
      "This Session's transaction has been rolled back "
  InvalidRequestError: This Session's transaction has been rolled back by a nested rollback() call.  To begin a new transaction, issue Session.rollback() first.
  }}}

  Traceback (most recent call last):
    File "neutron/tests/unit/vmware/test_nsx_plugin.py", line 946, in test_metadata_network_remove_rollback_on_failure
      webob.exc.HTTPOk.code)
    File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
      self.gen.next()
    File "neutron/tests/unit/test_l3_plugin.py", line 415, in router
      self._delete('routers', router['router']['id'])
    File "neutron/tests/unit/test_db_plugin.py", line 471, in _delete
      self.assertEqual(res.status_int, expected_code)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 348, in assertEqual
      self.assertThat(observed, matcher, message)
    File "/home/administrator/code/neutron/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 433, in assertThat
      raise mismatch_error
  MismatchError: 500 != 204

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1429015/+subscriptions


References