← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #28460

[Bug 1421497] [NEW] Gateway clear generates a TRACE - AttributeError in get_int_device_name in DVR routers

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

A recent change in the agent code have introduced this problem.

When a Gateway is cleared from the router, even though there are no existing floating IPs, the "external_gateway_removed" function in "agent.py" is calling "process_floatingips". 
That may be the reason for this failure.


Stderr: RTNETLINK answers: No such process

2015-02-11 23:12:15.307 2809 ERROR neutron.agent.l3.dvr [-] DVR: removed snat failed
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr Traceback (most recent call last):
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr   File "/opt/stack/new/neutron/neutron/agent/l3/dvr.py", line 197, in _snat_redirect_remove
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr     ns_ipd.route.delete_gateway(table=snat_idx)
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr   File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 415, in delete_gateway
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr     self._as_root(*args)
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr   File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 253, in _as_root
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr     kwargs.get('use_root_namespace', False))
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr   File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 83, in _as_root
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr     log_fail_as_error=self.log_fail_as_error)
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr   File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 95, in _execute
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr     log_fail_as_error=log_fail_as_error)
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr   File "/opt/stack/new/neutron/neutron/agent/linux/utils.py", line 83, in execute
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr     raise RuntimeError(m)
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr RuntimeError: 
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-1cfe7654-a669-4f73-a21d-d5110d7c0297', 'ip', 'route', 'del', 'default', 'dev', 'qr-467e8832-93', 'table', '547711270']
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr Exit code: 2
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr Stdout: 
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr Stderr: RTNETLINK answers: No such process
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr 
2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr 
2015-02-11 23:12:18.846 2809 ERROR neutron.agent.l3.agent [-] 'NoneType' object has no attribute 'get_int_device_name'
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent Traceback (most recent call last):
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent   File "/opt/stack/new/neutron/neutron/common/utils.py", line 342, in call
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent     return func(*args, **kwargs)
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent   File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 602, in process_router
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent     self._process_external(ri)
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent   File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 565, in _process_external
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent     self._process_external_gateway(ri)
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent   File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 503, in _process_external_gateway
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent     self.external_gateway_removed(ri, ri.ex_gw_port, interface_name)
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent   File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 905, in external_gateway_removed
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent     ri, ex_gw_port)
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent   File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 694, in _get_external_device_interface_name
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent     fip_int = ri.fip_ns.get_int_device_name(ri.router_id)
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent AttributeError: 'NoneType' object has no attribute 'get_int_device_name'
2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent 
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenpool.py", line 82, in _spawn_n_impl
    func(*args, **kwargs)
  File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 1137, in _process_router_update
    self._router_removed(update.id)
  File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 409, in _router_removed
    self.process_router(ri)
  File "/opt/stack/new/neutron/neutron/common/utils.py", line 345, in call
    self.logger(e)
  File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 82, in __exit__
    six.reraise(self.type_, self.value, self.tb)
  File "/opt/stack/new/neutron/neutron/common/utils.py", line 342, in call
    return func(*args, **kwargs)
  File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 602, in process_router
    self._process_external(ri)
  File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 565, in _process_external
    self._process_external_gateway(ri)
  File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 503, in _process_external_gateway
    self.external_gateway_removed(ri, ri.ex_gw_port, interface_name)
  File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 905, in external_gateway_removed
    ri, ex_gw_port)
  File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 694, in _get_external_device_interface_name
    fip_int = ri.fip_ns.get_int_device_name(ri.router_id)
AttributeError: 'NoneType' object has no attribute 'get_int_device_name'

** Affects: neutron
     Importance: Undecided
         Status: New


** Tags: l3-dvr-backlog

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1421497

Title:
  Gateway clear generates a TRACE - AttributeError in
  get_int_device_name in DVR routers

Status in OpenStack Neutron (virtual network service):
  New

Bug description:
  A recent change in the agent code have introduced this problem.

  When a Gateway is cleared from the router, even though there are no existing floating IPs, the "external_gateway_removed" function in "agent.py" is calling "process_floatingips". 
  That may be the reason for this failure.

  
  Stderr: RTNETLINK answers: No such process

  2015-02-11 23:12:15.307 2809 ERROR neutron.agent.l3.dvr [-] DVR: removed snat failed
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr Traceback (most recent call last):
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr   File "/opt/stack/new/neutron/neutron/agent/l3/dvr.py", line 197, in _snat_redirect_remove
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr     ns_ipd.route.delete_gateway(table=snat_idx)
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr   File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 415, in delete_gateway
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr     self._as_root(*args)
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr   File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 253, in _as_root
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr     kwargs.get('use_root_namespace', False))
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr   File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 83, in _as_root
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr     log_fail_as_error=self.log_fail_as_error)
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr   File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 95, in _execute
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr     log_fail_as_error=log_fail_as_error)
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr   File "/opt/stack/new/neutron/neutron/agent/linux/utils.py", line 83, in execute
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr     raise RuntimeError(m)
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr RuntimeError: 
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-1cfe7654-a669-4f73-a21d-d5110d7c0297', 'ip', 'route', 'del', 'default', 'dev', 'qr-467e8832-93', 'table', '547711270']
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr Exit code: 2
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr Stdout: 
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr Stderr: RTNETLINK answers: No such process
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr 
  2015-02-11 23:12:15.307 2809 TRACE neutron.agent.l3.dvr 
  2015-02-11 23:12:18.846 2809 ERROR neutron.agent.l3.agent [-] 'NoneType' object has no attribute 'get_int_device_name'
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent Traceback (most recent call last):
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent   File "/opt/stack/new/neutron/neutron/common/utils.py", line 342, in call
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent     return func(*args, **kwargs)
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent   File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 602, in process_router
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent     self._process_external(ri)
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent   File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 565, in _process_external
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent     self._process_external_gateway(ri)
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent   File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 503, in _process_external_gateway
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent     self.external_gateway_removed(ri, ri.ex_gw_port, interface_name)
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent   File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 905, in external_gateway_removed
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent     ri, ex_gw_port)
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent   File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 694, in _get_external_device_interface_name
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent     fip_int = ri.fip_ns.get_int_device_name(ri.router_id)
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent AttributeError: 'NoneType' object has no attribute 'get_int_device_name'
  2015-02-11 23:12:18.846 2809 TRACE neutron.agent.l3.agent 
  Traceback (most recent call last):
    File "/usr/local/lib/python2.7/dist-packages/eventlet/greenpool.py", line 82, in _spawn_n_impl
      func(*args, **kwargs)
    File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 1137, in _process_router_update
      self._router_removed(update.id)
    File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 409, in _router_removed
      self.process_router(ri)
    File "/opt/stack/new/neutron/neutron/common/utils.py", line 345, in call
      self.logger(e)
    File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 82, in __exit__
      six.reraise(self.type_, self.value, self.tb)
    File "/opt/stack/new/neutron/neutron/common/utils.py", line 342, in call
      return func(*args, **kwargs)
    File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 602, in process_router
      self._process_external(ri)
    File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 565, in _process_external
      self._process_external_gateway(ri)
    File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 503, in _process_external_gateway
      self.external_gateway_removed(ri, ri.ex_gw_port, interface_name)
    File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 905, in external_gateway_removed
      ri, ex_gw_port)
    File "/opt/stack/new/neutron/neutron/agent/l3/agent.py", line 694, in _get_external_device_interface_name
      fip_int = ri.fip_ns.get_int_device_name(ri.router_id)
  AttributeError: 'NoneType' object has no attribute 'get_int_device_name'

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1421497/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next