yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #78604
[Bug 1830295] [NEW] devstack py3 get_link_devices() KeyError: 'index'
Public bug reported:
devstack master with py3. openvswitch agent has suddenly stopped
working, with no change in config or environment (other than rebuilding
devstack). Stack trace below. For some reason (yet undetermined),
privileged.get_link_devices() now seems to be returning byte arrays
instead of strings as the dict keys:
>>> from neutron.privileged.agent.linux import ip_lib as privileged
>>> privileged.get_link_devices(None)[0].keys()
dict_keys([b'index', b'family', b'__align', b'header', b'flags', b'ifi_type', b'event', b'change', b'attrs'])
>>>
>From agent startup:
neutron-openvswitch-agent[42936]: ERROR neutron Traceback (most recent call last):
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/bin/neutron-openvswitch-agent", line 10, in <module>
neutron-openvswitch-agent[42936]: ERROR neutron sys.exit(main())
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/cmd/eventlet/plugins/ovs_neutron_agent.py", line 20, in main
neutron-openvswitch-agent[42936]: ERROR neutron agent_main.main()
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/main.py", line 47, in main
neutron-openvswitch-agent[42936]: ERROR neutron mod.main()
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/main.py", line 35, in main
neutron-openvswitch-agent[42936]: ERROR neutron 'neutron.plugins.ml2.drivers.openvswitch.agent.'
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/os_ken/base/app_manager.py", line 375, in run_apps
neutron-openvswitch-agent[42936]: ERROR neutron hub.joinall(services)
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/os_ken/lib/hub.py", line 102, in joinall
neutron-openvswitch-agent[42936]: ERROR neutron t.wait()
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/eventlet/greenthread.py", line 180, in wait
neutron-openvswitch-agent[42936]: ERROR neutron return self._exit_event.wait()
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/eventlet/event.py", line 132, in wait
neutron-openvswitch-agent[42936]: ERROR neutron current.throw(*self._exc)
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/eventlet/greenthread.py", line 219, in main
neutron-openvswitch-agent[42936]: ERROR neutron result = function(*args, **kwargs)
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/os_ken/lib/hub.py", line 64, in _launch
neutron-openvswitch-agent[42936]: ERROR neutron raise e
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/os_ken/lib/hub.py", line 59, in _launch
neutron-openvswitch-agent[42936]: ERROR neutron return func(*args, **kwargs)
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ovs_oskenapp.py", line 43, in agent_main_wrapper
neutron-openvswitch-agent[42936]: ERROR neutron LOG.exception("Agent main thread died of an exception")
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
neutron-openvswitch-agent[42936]: ERROR neutron self.force_reraise()
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
neutron-openvswitch-agent[42936]: ERROR neutron six.reraise(self.type_, self.value, self.tb)
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/six.py", line 693, in reraise
neutron-openvswitch-agent[42936]: ERROR neutron raise value
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ovs_oskenapp.py", line 40, in agent_main_wrapper
neutron-openvswitch-agent[42936]: ERROR neutron ovs_agent.main(bridge_classes)
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 2393, in main
neutron-openvswitch-agent[42936]: ERROR neutron validate_tunnel_config(cfg.CONF.AGENT.tunnel_types, cfg.CONF.OVS.local_ip)
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 2362, in validate_tunnel_config
neutron-openvswitch-agent[42936]: ERROR neutron validate_local_ip(local_ip)
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 2350, in validate_local_ip
neutron-openvswitch-agent[42936]: ERROR neutron if not ip_lib.IPWrapper().get_device_by_ip(local_ip):
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 187, in get_device_by_ip
neutron-openvswitch-agent[42936]: ERROR neutron devices = get_devices_with_ip(self.namespace, **kwargs)
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 1417, in get_devices_with_ip
neutron-openvswitch-agent[42936]: ERROR neutron for device in devices):
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 1417, in <genexpr>
neutron-openvswitch-agent[42936]: ERROR neutron for device in devices):
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 1388, in _parse_link_device
neutron-openvswitch-agent[42936]: ERROR neutron index=device['index'],
neutron-openvswitch-agent[42936]: ERROR neutron KeyError: 'index'
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1830295
Title:
devstack py3 get_link_devices() KeyError: 'index'
Status in neutron:
New
Bug description:
devstack master with py3. openvswitch agent has suddenly stopped
working, with no change in config or environment (other than
rebuilding devstack). Stack trace below. For some reason (yet
undetermined), privileged.get_link_devices() now seems to be returning
byte arrays instead of strings as the dict keys:
>>> from neutron.privileged.agent.linux import ip_lib as privileged
>>> privileged.get_link_devices(None)[0].keys()
dict_keys([b'index', b'family', b'__align', b'header', b'flags', b'ifi_type', b'event', b'change', b'attrs'])
>>>
From agent startup:
neutron-openvswitch-agent[42936]: ERROR neutron Traceback (most recent call last):
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/bin/neutron-openvswitch-agent", line 10, in <module>
neutron-openvswitch-agent[42936]: ERROR neutron sys.exit(main())
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/cmd/eventlet/plugins/ovs_neutron_agent.py", line 20, in main
neutron-openvswitch-agent[42936]: ERROR neutron agent_main.main()
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/main.py", line 47, in main
neutron-openvswitch-agent[42936]: ERROR neutron mod.main()
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/main.py", line 35, in main
neutron-openvswitch-agent[42936]: ERROR neutron 'neutron.plugins.ml2.drivers.openvswitch.agent.'
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/os_ken/base/app_manager.py", line 375, in run_apps
neutron-openvswitch-agent[42936]: ERROR neutron hub.joinall(services)
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/os_ken/lib/hub.py", line 102, in joinall
neutron-openvswitch-agent[42936]: ERROR neutron t.wait()
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/eventlet/greenthread.py", line 180, in wait
neutron-openvswitch-agent[42936]: ERROR neutron return self._exit_event.wait()
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/eventlet/event.py", line 132, in wait
neutron-openvswitch-agent[42936]: ERROR neutron current.throw(*self._exc)
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/eventlet/greenthread.py", line 219, in main
neutron-openvswitch-agent[42936]: ERROR neutron result = function(*args, **kwargs)
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/os_ken/lib/hub.py", line 64, in _launch
neutron-openvswitch-agent[42936]: ERROR neutron raise e
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/os_ken/lib/hub.py", line 59, in _launch
neutron-openvswitch-agent[42936]: ERROR neutron return func(*args, **kwargs)
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ovs_oskenapp.py", line 43, in agent_main_wrapper
neutron-openvswitch-agent[42936]: ERROR neutron LOG.exception("Agent main thread died of an exception")
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
neutron-openvswitch-agent[42936]: ERROR neutron self.force_reraise()
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
neutron-openvswitch-agent[42936]: ERROR neutron six.reraise(self.type_, self.value, self.tb)
neutron-openvswitch-agent[42936]: ERROR neutron File "/usr/local/lib/python3.6/dist-packages/six.py", line 693, in reraise
neutron-openvswitch-agent[42936]: ERROR neutron raise value
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ovs_oskenapp.py", line 40, in agent_main_wrapper
neutron-openvswitch-agent[42936]: ERROR neutron ovs_agent.main(bridge_classes)
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 2393, in main
neutron-openvswitch-agent[42936]: ERROR neutron validate_tunnel_config(cfg.CONF.AGENT.tunnel_types, cfg.CONF.OVS.local_ip)
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 2362, in validate_tunnel_config
neutron-openvswitch-agent[42936]: ERROR neutron validate_local_ip(local_ip)
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 2350, in validate_local_ip
neutron-openvswitch-agent[42936]: ERROR neutron if not ip_lib.IPWrapper().get_device_by_ip(local_ip):
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 187, in get_device_by_ip
neutron-openvswitch-agent[42936]: ERROR neutron devices = get_devices_with_ip(self.namespace, **kwargs)
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 1417, in get_devices_with_ip
neutron-openvswitch-agent[42936]: ERROR neutron for device in devices):
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 1417, in <genexpr>
neutron-openvswitch-agent[42936]: ERROR neutron for device in devices):
neutron-openvswitch-agent[42936]: ERROR neutron File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 1388, in _parse_link_device
neutron-openvswitch-agent[42936]: ERROR neutron index=device['index'],
neutron-openvswitch-agent[42936]: ERROR neutron KeyError: 'index'
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1830295/+subscriptions
Follow ups