| Thread Previous • Date Previous • Date Next • Thread Next |
Public bug reported:
Environment:
Stable/Liberty with neutron ML2 (Mechanism Driver is a custom asynchronous driver based off the Opendaylight V2 driver)
No agents or OVS bridges in use.
One controller and network node and two compute nodes.
When a VM fails to start on any compute node, nova removes the host
binding from the VM but doesn't send a port update to neutron notifying
it about the hostbinding change.
When the same error state VM is deleted, nova doesn't send any events to
neutron. As a result the driver thinks the port is still active, owned
by an existing VM and bound to the host properly.
NOTE: This behavior is only seen with VM's in the ERROR state, ports for
VMs in ACTIVE state are deleted properly.
$ nova show vm1
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | AUTO |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hostname | vm1 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-0000077a |
| OS-EXT-SRV-ATTR:kernel_id | 72d053dc-aae8-4559-a7c4-107c980bd674 |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | 3d25fd54-ffbc-4370-b971-2196a7963c24 |
| OS-EXT-SRV-ATTR:reservation_id | r-pujfofv9 |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | error |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2016-06-20T22:51:13Z |
| fault | {"message": "No valid host was found. There are not enough hosts available.", "code": 500, "details": " File \"/opt/stack/nova/nova/conductor/manager.py\", line 739, in build_instances |
| flavor | m1.small (2) |
| hostId | |
| id | 5d37f0b1-d3ac-4246-badc-b1a2aa6c3137 |
| image | cirros-0.3.4-x86_64-uec (50976ce1-c00e-4a80-9d72-6bf5d1abe54a) |
| key_name | - |
| metadata | {} |
| name | vm1 |
| net1 network | 30.0.0.231 |
| os-extended-volumes:volumes_attached | [] |
| security_groups | default |
| status | ERROR |
| tenant_id | 4c3ddd212d4f4af9b4d0d55e89b43125 |
| updated | 2016-06-20T22:51:23Z |
| user_id | 1e3e7e3addc14861afe177f285e43b40 |
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ neutron port-show 5efcfa1b-1300-4194-8229-11863f85d0a9
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | bxb-ds-50 |
| binding:profile | {} |
| binding:vif_details | {"port_filter": false, "vhostuser_ovs_plug": false, "vhostuser_socket": "/tmp/sock-fa163eaba0d2", "vhostuser_mode": "server"} |
| binding:vif_type | vhostuser |
| binding:vnic_type | normal |
| device_id | 5d37f0b1-d3ac-4246-badc-b1a2aa6c3137 |
| device_owner | compute:nova |
| dns_assignment | {"hostname": "host-30-0-0-231", "ip_address": "30.0.0.231", "fqdn": "host-30-0-0-231.openstacklocal."} |
| dns_name | |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "880aac1b-662e-4a4f-b640-7929d69a2b3d", "ip_address": "30.0.0.231"} |
| id | 5efcfa1b-1300-4194-8229-11863f85d0a9 |
| mac_address | fa:16:3e:ab:a0:d2 |
| name | |
| network_id | cf31ce68-4c5b-4e22-b725-dff0b6511a3d |
| port_security_enabled | True |
| security_groups | ae59a53e-38f2-471d-9de9-b8aea815dc27 |
| status | ACTIVE |
| tenant_id | 4c3ddd212d4f4af9b4d0d55e89b43125 |
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
$ nova delete vm1
Request to delete server vm1 has been accepted.
$ nova show vm1
ERROR (CommandError): No server with a name or ID of 'vm1' exists.
$ neutron port-show 5efcfa1b-1300-4194-8229-11863f85d0a9
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | bxb-ds-50 |
| binding:profile | {} |
| binding:vif_details | {"port_filter": false, "vhostuser_ovs_plug": false, "vhostuser_socket": "/tmp/sock-fa163eaba0d2", "vhostuser_mode": "server"} |
| binding:vif_type | vhostuser |
| binding:vnic_type | normal |
| device_id | 5d37f0b1-d3ac-4246-badc-b1a2aa6c3137 |
| device_owner | compute:nova |
| dns_assignment | {"hostname": "host-30-0-0-231", "ip_address": "30.0.0.231", "fqdn": "host-30-0-0-231.openstacklocal."} |
| dns_name | |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "880aac1b-662e-4a4f-b640-7929d69a2b3d", "ip_address": "30.0.0.231"} |
| id | 5efcfa1b-1300-4194-8229-11863f85d0a9 |
| mac_address | fa:16:3e:ab:a0:d2 |
| name | |
| network_id | cf31ce68-4c5b-4e22-b725-dff0b6511a3d |
| port_security_enabled | True |
| security_groups | ae59a53e-38f2-471d-9de9-b8aea815dc27 |
| status | ACTIVE |
| tenant_id | 4c3ddd212d4f4af9b4d0d55e89b43125 |
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1594604
Title:
Nova does not delete or update neutron port for failed VMs
Status in OpenStack Compute (nova):
New
Bug description:
Environment:
Stable/Liberty with neutron ML2 (Mechanism Driver is a custom asynchronous driver based off the Opendaylight V2 driver)
No agents or OVS bridges in use.
One controller and network node and two compute nodes.
When a VM fails to start on any compute node, nova removes the host
binding from the VM but doesn't send a port update to neutron
notifying it about the hostbinding change.
When the same error state VM is deleted, nova doesn't send any events
to neutron. As a result the driver thinks the port is still active,
owned by an existing VM and bound to the host properly.
NOTE: This behavior is only seen with VM's in the ERROR state, ports
for VMs in ACTIVE state are deleted properly.
$ nova show vm1
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | AUTO |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hostname | vm1 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-0000077a |
| OS-EXT-SRV-ATTR:kernel_id | 72d053dc-aae8-4559-a7c4-107c980bd674 |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | 3d25fd54-ffbc-4370-b971-2196a7963c24 |
| OS-EXT-SRV-ATTR:reservation_id | r-pujfofv9 |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | error |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2016-06-20T22:51:13Z |
| fault | {"message": "No valid host was found. There are not enough hosts available.", "code": 500, "details": " File \"/opt/stack/nova/nova/conductor/manager.py\", line 739, in build_instances |
| flavor | m1.small (2) |
| hostId | |
| id | 5d37f0b1-d3ac-4246-badc-b1a2aa6c3137 |
| image | cirros-0.3.4-x86_64-uec (50976ce1-c00e-4a80-9d72-6bf5d1abe54a) |
| key_name | - |
| metadata | {} |
| name | vm1 |
| net1 network | 30.0.0.231 |
| os-extended-volumes:volumes_attached | [] |
| security_groups | default |
| status | ERROR |
| tenant_id | 4c3ddd212d4f4af9b4d0d55e89b43125 |
| updated | 2016-06-20T22:51:23Z |
| user_id | 1e3e7e3addc14861afe177f285e43b40 |
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ neutron port-show 5efcfa1b-1300-4194-8229-11863f85d0a9
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | bxb-ds-50 |
| binding:profile | {} |
| binding:vif_details | {"port_filter": false, "vhostuser_ovs_plug": false, "vhostuser_socket": "/tmp/sock-fa163eaba0d2", "vhostuser_mode": "server"} |
| binding:vif_type | vhostuser |
| binding:vnic_type | normal |
| device_id | 5d37f0b1-d3ac-4246-badc-b1a2aa6c3137 |
| device_owner | compute:nova |
| dns_assignment | {"hostname": "host-30-0-0-231", "ip_address": "30.0.0.231", "fqdn": "host-30-0-0-231.openstacklocal."} |
| dns_name | |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "880aac1b-662e-4a4f-b640-7929d69a2b3d", "ip_address": "30.0.0.231"} |
| id | 5efcfa1b-1300-4194-8229-11863f85d0a9 |
| mac_address | fa:16:3e:ab:a0:d2 |
| name | |
| network_id | cf31ce68-4c5b-4e22-b725-dff0b6511a3d |
| port_security_enabled | True |
| security_groups | ae59a53e-38f2-471d-9de9-b8aea815dc27 |
| status | ACTIVE |
| tenant_id | 4c3ddd212d4f4af9b4d0d55e89b43125 |
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
$ nova delete vm1
Request to delete server vm1 has been accepted.
$ nova show vm1
ERROR (CommandError): No server with a name or ID of 'vm1' exists.
$ neutron port-show 5efcfa1b-1300-4194-8229-11863f85d0a9
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | bxb-ds-50 |
| binding:profile | {} |
| binding:vif_details | {"port_filter": false, "vhostuser_ovs_plug": false, "vhostuser_socket": "/tmp/sock-fa163eaba0d2", "vhostuser_mode": "server"} |
| binding:vif_type | vhostuser |
| binding:vnic_type | normal |
| device_id | 5d37f0b1-d3ac-4246-badc-b1a2aa6c3137 |
| device_owner | compute:nova |
| dns_assignment | {"hostname": "host-30-0-0-231", "ip_address": "30.0.0.231", "fqdn": "host-30-0-0-231.openstacklocal."} |
| dns_name | |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "880aac1b-662e-4a4f-b640-7929d69a2b3d", "ip_address": "30.0.0.231"} |
| id | 5efcfa1b-1300-4194-8229-11863f85d0a9 |
| mac_address | fa:16:3e:ab:a0:d2 |
| name | |
| network_id | cf31ce68-4c5b-4e22-b725-dff0b6511a3d |
| port_security_enabled | True |
| security_groups | ae59a53e-38f2-471d-9de9-b8aea815dc27 |
| status | ACTIVE |
| tenant_id | 4c3ddd212d4f4af9b4d0d55e89b43125 |
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1594604/+subscriptions
| Thread Previous • Date Previous • Date Next • Thread Next |
