← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1951623] [NEW] Error live migration vm with disabled port

 

Public bug reported:

We use neutron(stable/stein) ml2/ovs plugin and nova (stable/stein)
An error occurs in case of server migration with a disabled port(admin_state_up DWON)/
Here ports configuration
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| ID                                   | Name | MAC Address       | Fixed IP Addresses                                                          | Status |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| 09d5db66-1f89-41dd-a215-74888f3099f3 |      | fa:16:3e:98:30:50 | ip_address='10.1.1.1', subnet_id='a6786536-b67b-40a4-9470-e3b158a71dbc' | ACTIVE |
| df08c6b0-ca45-4985-8e5f-8fb95f904ac6 |      | fa:16:3e:5c:eb:f4 | ip_address='192.168.0.7', subnet_id='0bb936ed-c4a4-4a5d-be18-2794a73aea79'  | DOWN   |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
When we try migrate what vm
openstack server  migrate b4743fab-17e0-48af-8ad3-3b81fd05a968 --live cmp1
An error occurs in the pre live migration process.
The error occurs on the server from which the migration is performed
2021-11-18 17:34:28,910.910 2173136 WARNING nova.compute.manager [-] [instance: b4743fab-17e0-48af-8ad3-3b81fd05a968] Timed out waiting for events: [('network-vif-plugged', u'e09dca39-f62f-4a3b-a0f6-4d98edcd037e'), ('network-vif-plugged', u'e331b3d3-cf59-49a0-b531-590433523f6f')]. If these timeouts are a persistent issue it could mean the networking backend on host cmp1 does not support sending these events unless there are port binding host changes which does not happen at this point in the live migration process. You may need to disable the live_migration_wait_for_vif_plug option on host cmp1.: Timeout: 300 seconds

This happens because nova-compute is waiting for an event on each port, regardless of its initial state.
https://github.com/openstack/nova/blob/stable/stein/nova/compute/manager.py#L6767

But the neutron server does not send a message if the port is disabled.
https://github.com/openstack/neutron/blob/stable/stein/neutron/notifiers/nova.py#L207

2021-11-18 16:09:03,642.642 2916237 DEBUG neutron.notifiers.nova
[req-98b397eb-db00-4370-9510-b071c501a12e
b6ba9c75146a49829a7427a3e8cc3c10 192796e61c174f718d6147b129f3f2ff -
default default] Ignoring state change previous_port_status: DOWN
current_port_status: DOWN port_id e09dca39-f62f-4a3b-a0f6-4d98edcd037e
record_port_status_changed /usr/lib/python2.7/dist-
packages/neutron/notifiers/nova.py

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1951623

Title:
  Error live migration vm with disabled port

Status in OpenStack Compute (nova):
  New

Bug description:
  We use neutron(stable/stein) ml2/ovs plugin and nova (stable/stein)
  An error occurs in case of server migration with a disabled port(admin_state_up DWON)/
  Here ports configuration
  +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
  | ID                                   | Name | MAC Address       | Fixed IP Addresses                                                          | Status |
  +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
  | 09d5db66-1f89-41dd-a215-74888f3099f3 |      | fa:16:3e:98:30:50 | ip_address='10.1.1.1', subnet_id='a6786536-b67b-40a4-9470-e3b158a71dbc' | ACTIVE |
  | df08c6b0-ca45-4985-8e5f-8fb95f904ac6 |      | fa:16:3e:5c:eb:f4 | ip_address='192.168.0.7', subnet_id='0bb936ed-c4a4-4a5d-be18-2794a73aea79'  | DOWN   |
  +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
  When we try migrate what vm
  openstack server  migrate b4743fab-17e0-48af-8ad3-3b81fd05a968 --live cmp1
  An error occurs in the pre live migration process.
  The error occurs on the server from which the migration is performed
  2021-11-18 17:34:28,910.910 2173136 WARNING nova.compute.manager [-] [instance: b4743fab-17e0-48af-8ad3-3b81fd05a968] Timed out waiting for events: [('network-vif-plugged', u'e09dca39-f62f-4a3b-a0f6-4d98edcd037e'), ('network-vif-plugged', u'e331b3d3-cf59-49a0-b531-590433523f6f')]. If these timeouts are a persistent issue it could mean the networking backend on host cmp1 does not support sending these events unless there are port binding host changes which does not happen at this point in the live migration process. You may need to disable the live_migration_wait_for_vif_plug option on host cmp1.: Timeout: 300 seconds

  This happens because nova-compute is waiting for an event on each port, regardless of its initial state.
  https://github.com/openstack/nova/blob/stable/stein/nova/compute/manager.py#L6767

  But the neutron server does not send a message if the port is disabled.
  https://github.com/openstack/neutron/blob/stable/stein/neutron/notifiers/nova.py#L207

  2021-11-18 16:09:03,642.642 2916237 DEBUG neutron.notifiers.nova
  [req-98b397eb-db00-4370-9510-b071c501a12e
  b6ba9c75146a49829a7427a3e8cc3c10 192796e61c174f718d6147b129f3f2ff -
  default default] Ignoring state change previous_port_status: DOWN
  current_port_status: DOWN port_id e09dca39-f62f-4a3b-a0f6-4d98edcd037e
  record_port_status_changed /usr/lib/python2.7/dist-
  packages/neutron/notifiers/nova.py

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1951623/+subscriptions