yahoo-eng-team team mailing list archive

[Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>]

** Changed in: neutron
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1367881

Title:
  l2pop RPC code throwing an exception in fdb_chg_ip_tun()

Status in OpenStack Neutron (virtual network service):
  Fix Released

Bug description:
  I'm seeing an error in the l2pop code where it's failing to add a flow
  for the ARP entry responder.

  This is sometimes leading to DHCP failures for VMs, although a soft
  reboot typically fixes that problem.

  Here is the trace:

  2014-09-10 15:10:36.954 9351 ERROR neutron.agent.linux.ovs_lib [req-de0c2985-1fac-46a8-a42b-f0bad5a43805 None] OVS flows could not be applied on bridge br-tun
  2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib Traceback (most recent call last):
  2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 407, in _fdb_chg_ip 
  2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib     self.local_ip, self.local_vlan_map)
  2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/common/log.py", line 36, in wrapper
  2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib     return method(*args, **kwargs)
  2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/l2population_rpc.py", line 250, in fdb_chg_ip_tun
  2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib     for mac, ip in after:
  2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib TypeError: 'NoneType' object is not iterable
  2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib 
  2014-09-10 15:10:36.955 9351 ERROR oslo.messaging.rpc.dispatcher [req-de0c2985-1fac-46a8-a42b-f0bad5a43805 ] Exception during message handling: 'NoneType' object is not iterable
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/common/log.py", line 36, in wrapper
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher     return method(*args, **kwargs)
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/l2population_rpc.py", line 55, in update_fdb_entries
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher     self.fdb_update(context, fdb_entries)
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/common/log.py", line 36, in wrapper
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher     return method(*args, **kwargs)
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/l2population_rpc.py", line 212, in fdb_update
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher     getattr(self, method)(context, values)
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 407, in _fdb_chg_ip
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher     self.local_ip, self.local_vlan_map)
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/common/log.py", line 36, in wrapper
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher     return method(*args, **kwargs)
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/l2population_rpc.py", line 250, in fdb_chg_ip_tun
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher     for mac, ip in after:
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher TypeError: 'NoneType' object is not iterable
  2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher 
  2014-09-10 15:10:36.957 9351 ERROR oslo.messaging._drivers.common [req-de0c2985-1fac-46a8-a42b-f0bad5a43805 ] Returning exception 'NoneType' object is not iterable to caller

  I don't know this code well enough to suggest a fix - whether it's
  checking the return from agent_ports.items() better, or that there is
  a bug elsewhere, so any help would be appreciated.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1367881/+subscriptions