yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1612812] [NEW] OS-VIF: Interface missing from Route Object. (OVS)

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Mark Doffman <mjdoffma@xxxxxxxxxx>
Date: Fri, 12 Aug 2016 21:00:12 -0000
Reply-to: Bug 1612812 <1612812@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:


When trying to boot and delete instances (Rally) saw this stack trace:

2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 2075, in build_resources
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] yield resources
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 1919, in build_and_run_instance
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] block_device_info=block_device_info)
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2658, in spawn
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] write_to_disk=True)
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4693, in _get_guest_xml
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] context)
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4525, in _get_guest_config
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] flavor, virt_type, self._host)
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 507, in get_config
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] vif_obj = os_vif_util.nova_to_osvif_vif(vif)
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 371, in nova_to_osvif_vif
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] vifobj = func(vif)
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 276, in _nova_to_osvif_vif_ovs
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] vif_name=_get_vif_name(vif))
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 239, in _get_vif_instance
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] network=_nova_to_osvif_network(vif['network']),
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 205, in _nova_to_osvif_network
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] subnets=_nova_to_osvif_subnets(network['subnets']))
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 191, in _nova_to_osvif_subnets
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] objects=[_nova_to_osvif_subnet(subnet) for subnet in subnets])
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 173, in _nova_to_osvif_subnet
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] routes=_nova_to_osvif_routes(subnet['routes']))
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 157, in _nova_to_osvif_routes
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] objects=[_nova_to_osvif_route(route) for route in routes])
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 139, in _nova_to_osvif_route
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] interface=route['interface'])
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 307, in __init
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] setattr(self, key, kwargs[key])
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 72, in setter
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] field_value = field.coerce(self, name, value)
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_versionedobjects/fields.py", line 193, in coerce
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] return self.null(obj, attr)
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_versionedobjects/fields.py", line 171, in _null
2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] raise ValueError(("Field %s' cannot be None") % attr) 2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] ValueError: Fieldinterface' cannot be None

I cannot see where the 'interface' is set on the Nova 'Route' object.
The object only seems to be created in one place.
nova/network/neutronv2/api.py:2160

Version: Master
Networking: OVN ML2 driver. OVS VIF.

** Affects: nova
     Importance: Undecided
         Status: New

** Summary changed:

- OS-VIF Interface missing from Route Object. (OVS)
+ OS-VIF: Interface missing from Route Object. (OVS)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1612812

Title:
  OS-VIF: Interface missing from Route Object. (OVS)

Status in OpenStack Compute (nova):
  New

Bug description:
  
  When trying to boot and delete instances (Rally) saw this stack trace:

  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 2075, in build_resources
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] yield resources
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 1919, in build_and_run_instance
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] block_device_info=block_device_info)
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2658, in spawn
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] write_to_disk=True)
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4693, in _get_guest_xml
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] context)
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4525, in _get_guest_config
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] flavor, virt_type, self._host)
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 507, in get_config
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] vif_obj = os_vif_util.nova_to_osvif_vif(vif)
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 371, in nova_to_osvif_vif
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] vifobj = func(vif)
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 276, in _nova_to_osvif_vif_ovs
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] vif_name=_get_vif_name(vif))
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 239, in _get_vif_instance
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] network=_nova_to_osvif_network(vif['network']),
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 205, in _nova_to_osvif_network
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] subnets=_nova_to_osvif_subnets(network['subnets']))
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 191, in _nova_to_osvif_subnets
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] objects=[_nova_to_osvif_subnet(subnet) for subnet in subnets])
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 173, in _nova_to_osvif_subnet
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] routes=_nova_to_osvif_routes(subnet['routes']))
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 157, in _nova_to_osvif_routes
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] objects=[_nova_to_osvif_route(route) for route in routes])
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 139, in _nova_to_osvif_route
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] interface=route['interface'])
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 307, in __init
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] setattr(self, key, kwargs[key])
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 72, in setter
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] field_value = field.coerce(self, name, value)
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_versionedobjects/fields.py", line 193, in coerce
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] return self.null(obj, attr)
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_versionedobjects/fields.py", line 171, in _null
  2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] raise ValueError(("Field %s' cannot be None") % attr) 2016-08-12 15:21:58.859 38365 ERROR nova.compute.manager [instance: 47fbd50b-94ec-4395-884c-9131e0e3f335] ValueError: Fieldinterface' cannot be None

  I cannot see where the 'interface' is set on the Nova 'Route' object.
  The object only seems to be created in one place.
  nova/network/neutronv2/api.py:2160

  Version: Master
  Networking: OVN ML2 driver. OVS VIF.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1612812/+subscriptions
Follow ups

[Bug 1612812] Re: OS-VIF: Interface missing from Route Object. (OVS)
From: OpenStack Infra, 2016-08-16
[Bug 1612812] Re: OS-VIF: Interface missing from Route Object. (OVS)
From: OpenStack Infra, 2016-08-13
[Bug 1612812] Re: OS-VIF: Interface missing from Route Object. (OVS)
From: Matt Riedemann, 2016-08-12