yahoo-eng-team team mailing list archive

[OpenStack Infra <1728642@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/516371
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4f2f5eeb8bbed7385cafbcd8ba2acb39870f4941
Submitter: Zuul
Branch:    master

commit 4f2f5eeb8bbed7385cafbcd8ba2acb39870f4941
Author: yong sheng gong <gong.yongsheng@xxxxxxxxxxx>
Date:   Tue Oct 31 00:05:20 2017 +0800

    Delete ovs port if namespace is corrupted
    
    This patch isolates a corrupted dhcp
    namespace and protects the ovs from being
    blasted by thousands of dangling ovs
    ports created by dhcp agent.
    
    Change-Id: I80138b3c37f41a18dc488a306768b8e2fa299eda
    Closes-bug: #1728642


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1728642

Title:
  corrupted namespace blasted ovs bridge with thousands of dangling port

Status in neutron:
  Fix Released

Bug description:
  when dhcp namespace is corrupted somehow, ovs bridge will be blasted
  with thousands of dangling ports which are created by dhcp agent.

  the corrupted namespace will cause following exception:

  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp [req-db1e4f25-2263-49e9-ba5b-308ea9ccfdec - - - - -] Unable to plug DHCP port for network 0c59667a-433a-4e97-9568-07ee6210c98b. Releasing port.
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp Traceback (most recent call last):
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/dhcp.py", line 1407, in setup
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp     self.plug(network, port, interface_name)
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/dhcp.py", line 1375, in plug
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp     mtu=network.get('mtu'))
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 268, in plug
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp     bridge, namespace, prefix, mtu)
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 389, in plug_new
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp     namespace_obj.add_device_to_namespace(ns_dev)
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 232, in add_device_to_namespace
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp     device.link.set_netns(self.namespace)
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 516, in set_netns
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp     self._as_root([], ('set', self.name, 'netns', namespace))
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 364, in _as_root
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp     use_root_namespace=use_root_namespace)
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 100, in _as_root
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp     log_fail_as_error=self.log_fail_as_error)
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 109, in _execute
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp     log_fail_as_error=log_fail_as_error)
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 156, in execute
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp     raise ProcessExecutionError(msg, returncode=returncode)
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: RTNETLINK answers: Invalid argument
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp
  2017-10-30 14:12:35.347 6 ERROR neutron.agent.linux.dhcp
  2017-10-30 14:12:35.479 6 ERROR neutron.agent.linux.utils [req-29d446ad-eed5-47a0-bfc7-496dad2d35f2 - - - - -] Exit code: 2; Stdin: ; Stdout: ; Stderr: RTNETLINK answers: Invalid argument

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1728642/+subscriptions