← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1809136] [NEW] Unsupported VIF type unbound convert '_nova_to_osvif_vif_unbound' on compute restart

 

Public bug reported:

This is a variant of an existing bug:

- https://bugs.launchpad.net/nova/+bug/1738373 tracks a similar
exception ('_nova_to_osvif_vif_binding_failed') on compute startup.

There are also two other closely related bugs:

- https://bugs.launchpad.net/nova/+bug/1783917 tracks this same exception ('_nova_to_osvif_vif_unbound') but for live migrations
- https://bugs.launchpad.net/nova/+bug/1784579 tracks a similar exception ('_nova_to_osvif_vif_binding_failed') but for live migration

In addition, there are a few bugs which are likely the root cause of all
of the above issues (and this one) in the first place:

- https://bugs.launchpad.net/nova/+bug/1751923

In this instance, as with bug 1738373, we are unable to start nova-
compute service on compute node due to an os-vif invoked error.

nova-compute.log on compute shows:

2018-05-12 16:42:47.323 305978 INFO os_vif [req-0a72cdea-843a-4932-b8a0-bc24c2f21d9f - - - - -] Successfully plugged vif VIFBridge(active=True,address=fa:16:3e:41:a9:2c,bridge_name='qbr8d027ff4-23',has_traffic_filtering=True,id=8d027ff4-2328-47df-9f9a-2c1a9914a83b,network=Network(9a98b244-b1d2-46b3-ab0e-be8456e3a984),plugin='ovs',port_profile=VIFPortProfileBase,preserve_on_delete=False,vif_name='tap8d027ff4-23')
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service [req-0a72cdea-843a-4932-b8a0-bc24c2f21d9f - - - - -] Error starting thread.
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service Traceback (most recent call last):
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 708, in run_service
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     service.start()
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/nova/service.py", line 117, in start
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     self.manager.init_host()
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1154, in init_host
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     self._init_instance(context, instance)
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 957, in _init_instance
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     self.driver.plug_vifs(instance, net_info)
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 703, in plug_vifs
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     self.vif_driver.plug(instance, vif)
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 771, in plug
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     vif_obj = os_vif_util.nova_to_osvif_vif(vif)
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 408, in nova_to_osvif_vif
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     {'type': vif['type'], 'func': funcname})
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service NovaException: Unsupported VIF type unbound convert '_nova_to_osvif_vif_unbound'
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service

Inspecting the available ports shows the port does exist, so this looks
like a caching issue.

[stack@director:~]$ neutron port-list | grep fa:16:3e:41:a9:2c
| 8d027ff4-2328-47df-9f9a-2c1a9914a83b |                                                                                                        | fa:16:3e:41:a9:2c | {"subnet_id": "1f5ed9bc-aa7d-49bd-ac48-23b430fc0eb4", "ip_address": "172.19.9.17"}     |
[stack@director:~]$ neutron port-show 8d027ff4-2328-47df-9f9a-2c1a9914a83b
+-----------------------+------------------------------------------------------------------------------------+
| Field                 | Value                                                                              |
+-----------------------+------------------------------------------------------------------------------------+
| admin_state_up        | True                                                                               |
| allowed_address_pairs |                                                                                    |
| binding:host_id       | overcloud-compute-7.localdomain                                                    |
| binding:profile       | {}                                                                                 |
| binding:vif_details   | {"port_filter": true, "ovs_hybrid_plug": true}                                     |
| binding:vif_type      | ovs                                                                                |
| binding:vnic_type     | normal                                                                             |
| created_at            | 2017-10-31T12:31:45Z                                                               |
| description           |                                                                                    |
| device_id             | b4ef4d0b-9e39-4741-a2dd-7fd7c066d13b                                               |
| device_owner          | compute:nova                                                                       |
| extra_dhcp_opts       |                                                                                    |
| fixed_ips             | {"subnet_id": "1f5ed9bc-aa7d-49bd-ac48-23b430fc0eb4", "ip_address": "172.19.9.17"} |
| id                    | 8d027ff4-2328-47df-9f9a-2c1a9914a83b                                               |
| mac_address           | fa:16:3e:41:a9:2c                                                                  |
| name                  |                                                                                    |
| network_id            | 9a98b244-b1d2-46b3-ab0e-be8456e3a984                                               |
| port_security_enabled | True                                                                               |
| project_id            | 3b2049626c954cdc9147beee2d34b441                                                   |
| qos_policy_id         |                                                                                    |
| revision_number       | 184                                                                                |
| security_groups       | 97aa0764-c0b5-47d1-88b2-285673d46a31                                               |
|                       | c7addc13-5a77-4322-953a-9d89d42468e6                                               |
|                       | cecdad42-7c78-45e7-9ec2-fef1086dbb7e                                               |
|                       | de0a6da8-c44e-475f-90fd-1fb625840c52                                               |
| status                | ACTIVE                                                                             |
| tenant_id             | 3b2049626c954cdc9147beee2d34b441                                                   |
| updated_at            | 2018-05-12T15:37:46Z                                                               |
+-----------------------+------------------------------------------------------------------------------------+

We should figure out why the invalid cache is getting saved, but we're
going to track that effort separately. For now, we should just focus on
letting the service start, putting instances with errors like this into
error state.

This was originally reported here
https://bugzilla.redhat.com/show_bug.cgi?id=1578028

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1809136

Title:
  Unsupported VIF type unbound convert '_nova_to_osvif_vif_unbound' on
  compute restart

Status in OpenStack Compute (nova):
  New

Bug description:
  This is a variant of an existing bug:

  - https://bugs.launchpad.net/nova/+bug/1738373 tracks a similar
  exception ('_nova_to_osvif_vif_binding_failed') on compute startup.

  There are also two other closely related bugs:

  - https://bugs.launchpad.net/nova/+bug/1783917 tracks this same exception ('_nova_to_osvif_vif_unbound') but for live migrations
  - https://bugs.launchpad.net/nova/+bug/1784579 tracks a similar exception ('_nova_to_osvif_vif_binding_failed') but for live migration

  In addition, there are a few bugs which are likely the root cause of
  all of the above issues (and this one) in the first place:

  - https://bugs.launchpad.net/nova/+bug/1751923

  In this instance, as with bug 1738373, we are unable to start nova-
  compute service on compute node due to an os-vif invoked error.

  nova-compute.log on compute shows:

  2018-05-12 16:42:47.323 305978 INFO os_vif [req-0a72cdea-843a-4932-b8a0-bc24c2f21d9f - - - - -] Successfully plugged vif VIFBridge(active=True,address=fa:16:3e:41:a9:2c,bridge_name='qbr8d027ff4-23',has_traffic_filtering=True,id=8d027ff4-2328-47df-9f9a-2c1a9914a83b,network=Network(9a98b244-b1d2-46b3-ab0e-be8456e3a984),plugin='ovs',port_profile=VIFPortProfileBase,preserve_on_delete=False,vif_name='tap8d027ff4-23')
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service [req-0a72cdea-843a-4932-b8a0-bc24c2f21d9f - - - - -] Error starting thread.
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service Traceback (most recent call last):
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 708, in run_service
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     service.start()
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/nova/service.py", line 117, in start
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     self.manager.init_host()
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1154, in init_host
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     self._init_instance(context, instance)
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 957, in _init_instance
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     self.driver.plug_vifs(instance, net_info)
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 703, in plug_vifs
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     self.vif_driver.plug(instance, vif)
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 771, in plug
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     vif_obj = os_vif_util.nova_to_osvif_vif(vif)
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 408, in nova_to_osvif_vif
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service     {'type': vif['type'], 'func': funcname})
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service NovaException: Unsupported VIF type unbound convert '_nova_to_osvif_vif_unbound'
  2018-05-12 16:42:47.369 305978 ERROR oslo_service.service

  Inspecting the available ports shows the port does exist, so this
  looks like a caching issue.

  [stack@director:~]$ neutron port-list | grep fa:16:3e:41:a9:2c
  | 8d027ff4-2328-47df-9f9a-2c1a9914a83b |                                                                                                        | fa:16:3e:41:a9:2c | {"subnet_id": "1f5ed9bc-aa7d-49bd-ac48-23b430fc0eb4", "ip_address": "172.19.9.17"}     |
  [stack@director:~]$ neutron port-show 8d027ff4-2328-47df-9f9a-2c1a9914a83b
  +-----------------------+------------------------------------------------------------------------------------+
  | Field                 | Value                                                                              |
  +-----------------------+------------------------------------------------------------------------------------+
  | admin_state_up        | True                                                                               |
  | allowed_address_pairs |                                                                                    |
  | binding:host_id       | overcloud-compute-7.localdomain                                                    |
  | binding:profile       | {}                                                                                 |
  | binding:vif_details   | {"port_filter": true, "ovs_hybrid_plug": true}                                     |
  | binding:vif_type      | ovs                                                                                |
  | binding:vnic_type     | normal                                                                             |
  | created_at            | 2017-10-31T12:31:45Z                                                               |
  | description           |                                                                                    |
  | device_id             | b4ef4d0b-9e39-4741-a2dd-7fd7c066d13b                                               |
  | device_owner          | compute:nova                                                                       |
  | extra_dhcp_opts       |                                                                                    |
  | fixed_ips             | {"subnet_id": "1f5ed9bc-aa7d-49bd-ac48-23b430fc0eb4", "ip_address": "172.19.9.17"} |
  | id                    | 8d027ff4-2328-47df-9f9a-2c1a9914a83b                                               |
  | mac_address           | fa:16:3e:41:a9:2c                                                                  |
  | name                  |                                                                                    |
  | network_id            | 9a98b244-b1d2-46b3-ab0e-be8456e3a984                                               |
  | port_security_enabled | True                                                                               |
  | project_id            | 3b2049626c954cdc9147beee2d34b441                                                   |
  | qos_policy_id         |                                                                                    |
  | revision_number       | 184                                                                                |
  | security_groups       | 97aa0764-c0b5-47d1-88b2-285673d46a31                                               |
  |                       | c7addc13-5a77-4322-953a-9d89d42468e6                                               |
  |                       | cecdad42-7c78-45e7-9ec2-fef1086dbb7e                                               |
  |                       | de0a6da8-c44e-475f-90fd-1fb625840c52                                               |
  | status                | ACTIVE                                                                             |
  | tenant_id             | 3b2049626c954cdc9147beee2d34b441                                                   |
  | updated_at            | 2018-05-12T15:37:46Z                                                               |
  +-----------------------+------------------------------------------------------------------------------------+

  We should figure out why the invalid cache is getting saved, but we're
  going to track that effort separately. For now, we should just focus
  on letting the service start, putting instances with errors like this
  into error state.

  This was originally reported here
  https://bugzilla.redhat.com/show_bug.cgi?id=1578028

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1809136/+subscriptions


Follow ups