yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1460494] Re: neutron-ovs-cleanup failing to start with ovs bonding

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Assaf Muller <amuller@xxxxxxxxxx>
Date: Wed, 16 Dec 2015 21:17:06 -0000
Reply-to: Bug 1460494 <1460494@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1460494

Title:
  neutron-ovs-cleanup failing to start with ovs bonding

Status in neutron:
  Fix Released

Bug description:
  System is running Openstack Kilo:

  [root@network01 ~]# cat /etc/redhat-release
  CentOS Linux release 7.1.1503 (Core)

  [root@network01 ~]# uname -r
  3.10.0-229.4.2.el7.x86_64

  [root@network01 ~]# rpm -qa|grep neutron
  openstack-neutron-ml2-2015.1.0-1.el7.noarch
  openstack-neutron-common-2015.1.0-1.el7.noarch
  openstack-neutron-openvswitch-2015.1.0-1.el7.noarch
  python-neutronclient-2.3.11-1.el7.noarch
  python-neutron-2015.1.0-1.el7.noarch
  openstack-neutron-2015.1.0-1.el7.noarch

  Issue is with python-neutron-2015.1.0-1.el7.noarch line 316 of:

  /usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py

  get_vif_ports function is expecting that each ovs port will have an
  interface with a matching name, i.e. ovs port eth0 must have ovs
  Interface with name eth0. This does not work for bonded interfaces
  created within ovs as port bond2 may have Interface eth0 and Interface
  eth1 as an example.

  2015-05-29 16:53:06.034 2833 INFO neutron.common.config [-] Logging enabled!
  2015-05-29 16:53:06.039 2833 INFO neutron.common.config [-] /usr/bin/neutron-ovs-cleanup version 2015.1.0
  2015-05-29 16:53:06.502 2833 ERROR neutron.agent.ovsdb.impl_vsctl [-] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=external_ids', 'list', 'Interface', 'bond2'].
  2015-05-29 16:53:06.502 2833 TRACE neutron.agent.ovsdb.impl_vsctl Traceback (most recent call last):
  2015-05-29 16:53:06.502 2833 TRACE neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
  2015-05-29 16:53:06.502 2833 TRACE neutron.agent.ovsdb.impl_vsctl     log_fail_as_error=False).rstrip()
  2015-05-29 16:53:06.502 2833 TRACE neutron.agent.ovsdb.impl_vsctl   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 137, in execute
  2015-05-29 16:53:06.502 2833 TRACE neutron.agent.ovsdb.impl_vsctl     raise RuntimeError(m)
  2015-05-29 16:53:06.502 2833 TRACE neutron.agent.ovsdb.impl_vsctl RuntimeError:
  2015-05-29 16:53:06.502 2833 TRACE neutron.agent.ovsdb.impl_vsctl Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=external_ids', 'list', 'Interface', 'bond2']
  2015-05-29 16:53:06.502 2833 TRACE neutron.agent.ovsdb.impl_vsctl Exit code: 1
  2015-05-29 16:53:06.502 2833 TRACE neutron.agent.ovsdb.impl_vsctl Stdin:
  2015-05-29 16:53:06.502 2833 TRACE neutron.agent.ovsdb.impl_vsctl Stdout:
  2015-05-29 16:53:06.502 2833 TRACE neutron.agent.ovsdb.impl_vsctl Stderr: ovs-vsctl: no row "bond2" in table Interface
  2015-05-29 16:53:06.502 2833 TRACE neutron.agent.ovsdb.impl_vsctl
  2015-05-29 16:53:06.502 2833 TRACE neutron.agent.ovsdb.impl_vsctl
  2015-05-29 16:53:06.503 2833 CRITICAL neutron [-] RuntimeError:
  Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=external_ids', 'list', 'Interface', 'bond2']
  Exit code: 1
  Stdin:
  Stdout:
  Stderr: ovs-vsctl: no row "bond2" in table Interface

  2015-05-29 16:53:06.503 2833 TRACE neutron Traceback (most recent call last):
  2015-05-29 16:53:06.503 2833 TRACE neutron   File "/usr/bin/neutron-ovs-cleanup", line 10, in <module>
  2015-05-29 16:53:06.503 2833 TRACE neutron     sys.exit(main())
  2015-05-29 16:53:06.503 2833 TRACE neutron   File "/usr/lib/python2.7/site-packages/neutron/cmd/ovs_cleanup.py", line 100, in main
  2015-05-29 16:53:06.503 2833 TRACE neutron     ports = collect_neutron_ports(available_configuration_bridges)
  2015-05-29 16:53:06.503 2833 TRACE neutron   File "/usr/lib/python2.7/site-packages/neutron/cmd/ovs_cleanup.py", line 60, in collect_neutron_ports
  2015-05-29 16:53:06.503 2833 TRACE neutron     ports += [port.port_name for port in ovs.get_vif_ports()]
  2015-05-29 16:53:06.503 2833 TRACE neutron   File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 316, in get_vif_ports
  2015-05-29 16:53:06.503 2833 TRACE neutron     check_error=True)
  2015-05-29 16:53:06.503 2833 TRACE neutron   File "/usr/lib/python2.7/site-packages/neutron/agent/common/ovs_lib.py", line 139, in db_get_val
  2015-05-29 16:53:06.503 2833 TRACE neutron     check_error=check_error)
  2015-05-29 16:53:06.503 2833 TRACE neutron   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 83, in execute
  2015-05-29 16:53:06.503 2833 TRACE neutron     txn.add(self)
  2015-05-29 16:53:06.503 2833 TRACE neutron   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/api.py", line 68, in __exit__
  2015-05-29 16:53:06.503 2833 TRACE neutron     self.result = self.commit()
  2015-05-29 16:53:06.503 2833 TRACE neutron   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 50, in commit
  2015-05-29 16:53:06.503 2833 TRACE neutron     res = self.run_vsctl(args)
  2015-05-29 16:53:06.503 2833 TRACE neutron   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 70, in run_vsctl
  2015-05-29 16:53:06.503 2833 TRACE neutron     ctxt.reraise = False
  2015-05-29 16:53:06.503 2833 TRACE neutron   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
  2015-05-29 16:53:06.503 2833 TRACE neutron     six.reraise(self.type_, self.value, self.tb)
  2015-05-29 16:53:06.503 2833 TRACE neutron   File "/usr/lib/python2.7/site-packages/neutron/agent/ovsdb/impl_vsctl.py", line 63, in run_vsctl
  2015-05-29 16:53:06.503 2833 TRACE neutron     log_fail_as_error=False).rstrip()
  2015-05-29 16:53:06.503 2833 TRACE neutron   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 137, in execute
  2015-05-29 16:53:06.503 2833 TRACE neutron     raise RuntimeError(m)
  2015-05-29 16:53:06.503 2833 TRACE neutron RuntimeError:
  2015-05-29 16:53:06.503 2833 TRACE neutron Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=external_ids', 'list', 'Interface', 'bond2']
  2015-05-29 16:53:06.503 2833 TRACE neutron Exit code: 1
  2015-05-29 16:53:06.503 2833 TRACE neutron Stdin:
  2015-05-29 16:53:06.503 2833 TRACE neutron Stdout:
  2015-05-29 16:53:06.503 2833 TRACE neutron Stderr: ovs-vsctl: no row "bond2" in table Interface
  2015-05-29 16:53:06.503 2833 TRACE neutron
  2015-05-29 16:53:06.503 2833 TRACE neutron

  ovs_lib.py should be updated so that for an ovs bond port all
  interfaces belonging to the bond are looked up without relying on the
  name of the Interface to be matching.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1460494/+subscriptions
References

[Bug 1460494] [NEW] neutron-ovs-cleanup failing to start with ovs bonding
From: Ross Annetts, 2015-06-01