yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #76395
[Bug 1809136] [NEW] Unsupported VIF type unbound convert '_nova_to_osvif_vif_unbound' on compute restart
Public bug reported:
This is a variant of an existing bug:
- https://bugs.launchpad.net/nova/+bug/1738373 tracks a similar
exception ('_nova_to_osvif_vif_binding_failed') on compute startup.
There are also two other closely related bugs:
- https://bugs.launchpad.net/nova/+bug/1783917 tracks this same exception ('_nova_to_osvif_vif_unbound') but for live migrations
- https://bugs.launchpad.net/nova/+bug/1784579 tracks a similar exception ('_nova_to_osvif_vif_binding_failed') but for live migration
In addition, there are a few bugs which are likely the root cause of all
of the above issues (and this one) in the first place:
- https://bugs.launchpad.net/nova/+bug/1751923
In this instance, as with bug 1738373, we are unable to start nova-
compute service on compute node due to an os-vif invoked error.
nova-compute.log on compute shows:
2018-05-12 16:42:47.323 305978 INFO os_vif [req-0a72cdea-843a-4932-b8a0-bc24c2f21d9f - - - - -] Successfully plugged vif VIFBridge(active=True,address=fa:16:3e:41:a9:2c,bridge_name='qbr8d027ff4-23',has_traffic_filtering=True,id=8d027ff4-2328-47df-9f9a-2c1a9914a83b,network=Network(9a98b244-b1d2-46b3-ab0e-be8456e3a984),plugin='ovs',port_profile=VIFPortProfileBase,preserve_on_delete=False,vif_name='tap8d027ff4-23')
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service [req-0a72cdea-843a-4932-b8a0-bc24c2f21d9f - - - - -] Error starting thread.
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service Traceback (most recent call last):
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 708, in run_service
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service service.start()
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/service.py", line 117, in start
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service self.manager.init_host()
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1154, in init_host
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service self._init_instance(context, instance)
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 957, in _init_instance
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service self.driver.plug_vifs(instance, net_info)
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 703, in plug_vifs
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service self.vif_driver.plug(instance, vif)
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 771, in plug
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service vif_obj = os_vif_util.nova_to_osvif_vif(vif)
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 408, in nova_to_osvif_vif
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service {'type': vif['type'], 'func': funcname})
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service NovaException: Unsupported VIF type unbound convert '_nova_to_osvif_vif_unbound'
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service
Inspecting the available ports shows the port does exist, so this looks
like a caching issue.
[stack@director:~]$ neutron port-list | grep fa:16:3e:41:a9:2c
| 8d027ff4-2328-47df-9f9a-2c1a9914a83b | | fa:16:3e:41:a9:2c | {"subnet_id": "1f5ed9bc-aa7d-49bd-ac48-23b430fc0eb4", "ip_address": "172.19.9.17"} |
[stack@director:~]$ neutron port-show 8d027ff4-2328-47df-9f9a-2c1a9914a83b
+-----------------------+------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | overcloud-compute-7.localdomain |
| binding:profile | {} |
| binding:vif_details | {"port_filter": true, "ovs_hybrid_plug": true} |
| binding:vif_type | ovs |
| binding:vnic_type | normal |
| created_at | 2017-10-31T12:31:45Z |
| description | |
| device_id | b4ef4d0b-9e39-4741-a2dd-7fd7c066d13b |
| device_owner | compute:nova |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "1f5ed9bc-aa7d-49bd-ac48-23b430fc0eb4", "ip_address": "172.19.9.17"} |
| id | 8d027ff4-2328-47df-9f9a-2c1a9914a83b |
| mac_address | fa:16:3e:41:a9:2c |
| name | |
| network_id | 9a98b244-b1d2-46b3-ab0e-be8456e3a984 |
| port_security_enabled | True |
| project_id | 3b2049626c954cdc9147beee2d34b441 |
| qos_policy_id | |
| revision_number | 184 |
| security_groups | 97aa0764-c0b5-47d1-88b2-285673d46a31 |
| | c7addc13-5a77-4322-953a-9d89d42468e6 |
| | cecdad42-7c78-45e7-9ec2-fef1086dbb7e |
| | de0a6da8-c44e-475f-90fd-1fb625840c52 |
| status | ACTIVE |
| tenant_id | 3b2049626c954cdc9147beee2d34b441 |
| updated_at | 2018-05-12T15:37:46Z |
+-----------------------+------------------------------------------------------------------------------------+
We should figure out why the invalid cache is getting saved, but we're
going to track that effort separately. For now, we should just focus on
letting the service start, putting instances with errors like this into
error state.
This was originally reported here
https://bugzilla.redhat.com/show_bug.cgi?id=1578028
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1809136
Title:
Unsupported VIF type unbound convert '_nova_to_osvif_vif_unbound' on
compute restart
Status in OpenStack Compute (nova):
New
Bug description:
This is a variant of an existing bug:
- https://bugs.launchpad.net/nova/+bug/1738373 tracks a similar
exception ('_nova_to_osvif_vif_binding_failed') on compute startup.
There are also two other closely related bugs:
- https://bugs.launchpad.net/nova/+bug/1783917 tracks this same exception ('_nova_to_osvif_vif_unbound') but for live migrations
- https://bugs.launchpad.net/nova/+bug/1784579 tracks a similar exception ('_nova_to_osvif_vif_binding_failed') but for live migration
In addition, there are a few bugs which are likely the root cause of
all of the above issues (and this one) in the first place:
- https://bugs.launchpad.net/nova/+bug/1751923
In this instance, as with bug 1738373, we are unable to start nova-
compute service on compute node due to an os-vif invoked error.
nova-compute.log on compute shows:
2018-05-12 16:42:47.323 305978 INFO os_vif [req-0a72cdea-843a-4932-b8a0-bc24c2f21d9f - - - - -] Successfully plugged vif VIFBridge(active=True,address=fa:16:3e:41:a9:2c,bridge_name='qbr8d027ff4-23',has_traffic_filtering=True,id=8d027ff4-2328-47df-9f9a-2c1a9914a83b,network=Network(9a98b244-b1d2-46b3-ab0e-be8456e3a984),plugin='ovs',port_profile=VIFPortProfileBase,preserve_on_delete=False,vif_name='tap8d027ff4-23')
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service [req-0a72cdea-843a-4932-b8a0-bc24c2f21d9f - - - - -] Error starting thread.
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service Traceback (most recent call last):
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 708, in run_service
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service service.start()
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/service.py", line 117, in start
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service self.manager.init_host()
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1154, in init_host
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service self._init_instance(context, instance)
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 957, in _init_instance
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service self.driver.plug_vifs(instance, net_info)
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 703, in plug_vifs
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service self.vif_driver.plug(instance, vif)
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 771, in plug
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service vif_obj = os_vif_util.nova_to_osvif_vif(vif)
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/nova/network/os_vif_util.py", line 408, in nova_to_osvif_vif
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service {'type': vif['type'], 'func': funcname})
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service NovaException: Unsupported VIF type unbound convert '_nova_to_osvif_vif_unbound'
2018-05-12 16:42:47.369 305978 ERROR oslo_service.service
Inspecting the available ports shows the port does exist, so this
looks like a caching issue.
[stack@director:~]$ neutron port-list | grep fa:16:3e:41:a9:2c
| 8d027ff4-2328-47df-9f9a-2c1a9914a83b | | fa:16:3e:41:a9:2c | {"subnet_id": "1f5ed9bc-aa7d-49bd-ac48-23b430fc0eb4", "ip_address": "172.19.9.17"} |
[stack@director:~]$ neutron port-show 8d027ff4-2328-47df-9f9a-2c1a9914a83b
+-----------------------+------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | overcloud-compute-7.localdomain |
| binding:profile | {} |
| binding:vif_details | {"port_filter": true, "ovs_hybrid_plug": true} |
| binding:vif_type | ovs |
| binding:vnic_type | normal |
| created_at | 2017-10-31T12:31:45Z |
| description | |
| device_id | b4ef4d0b-9e39-4741-a2dd-7fd7c066d13b |
| device_owner | compute:nova |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "1f5ed9bc-aa7d-49bd-ac48-23b430fc0eb4", "ip_address": "172.19.9.17"} |
| id | 8d027ff4-2328-47df-9f9a-2c1a9914a83b |
| mac_address | fa:16:3e:41:a9:2c |
| name | |
| network_id | 9a98b244-b1d2-46b3-ab0e-be8456e3a984 |
| port_security_enabled | True |
| project_id | 3b2049626c954cdc9147beee2d34b441 |
| qos_policy_id | |
| revision_number | 184 |
| security_groups | 97aa0764-c0b5-47d1-88b2-285673d46a31 |
| | c7addc13-5a77-4322-953a-9d89d42468e6 |
| | cecdad42-7c78-45e7-9ec2-fef1086dbb7e |
| | de0a6da8-c44e-475f-90fd-1fb625840c52 |
| status | ACTIVE |
| tenant_id | 3b2049626c954cdc9147beee2d34b441 |
| updated_at | 2018-05-12T15:37:46Z |
+-----------------------+------------------------------------------------------------------------------------+
We should figure out why the invalid cache is getting saved, but we're
going to track that effort separately. For now, we should just focus
on letting the service start, putting instances with errors like this
into error state.
This was originally reported here
https://bugzilla.redhat.com/show_bug.cgi?id=1578028
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1809136/+subscriptions
Follow ups
