yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2051907] [NEW] Failed to live migrate instance to another host

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: chusnulchikam <2051907@xxxxxxxxxxxxxxxxxx>
Date: Thu, 01 Feb 2024 05:25:17 -0000
Reply-to: Bug 2051907 <2051907@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi [None req-d5465c38-0025-4c44-ac5b-5261cbe9d469 645810eb66cb45f38220ad1ade8447fb 073b3acb79724ec0a8246c8984489c58 - - default default] Unexpected exception in API method: nova.exception_Remote.MigrationError_Remote: Migration error: rule:create_port_binding is disallowed by policy
Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']
Traceback (most recent call last):

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
    ret = obj(*args, **kwargs)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response
    exception_handler_v20(status_code, error_body)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20
    raise client_exc(message=error_message,

neutronclient.common.exceptions.Forbidden: rule:create_port_binding is disallowed by policy
Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 505, in _live_migrate
    task.execute()

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/base.py", line 25, in wrap
    with excutils.save_and_reraise_exception():

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
    self.force_reraise()

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
    raise self.value

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/base.py", line 23, in wrap
    return original(self)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/base.py", line 40, in execute
    return self._execute()

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 100, in _execute
    self.destination, dest_node, self.limits = self._find_destination()

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 550, in _find_destination
    self._call_livem_checks_on_host(host, provider_mapping)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 370, in _call_livem_checks_on_host
    bindings = self._bind_ports_on_destination(

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 411, in _bind_ports_on_destination
    bindings = self.network_api.bind_ports_to_host(

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 1541, in bind_ports_to_host
    binding = client.create_port_binding(port_id, data)['binding']

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
    ret = obj(*args, **kwargs)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 837, in create_port_binding
    return self.post(self.port_bindings_path % port_id, body=body)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
    ret = obj(*args, **kwargs)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 361, in post
    return self.do_request("POST", action, body=body,

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
    ret = obj(*args, **kwargs)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 297, in do_request
    self._handle_fault_response(status_code, replybody, resp)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 214, in wrapper
    raise exception.Forbidden(str(e))

nova.exception.Forbidden: rule:create_port_binding is disallowed by policy
Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
    res = self.dispatcher.dispatch(message)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
    return self._do_dispatch(endpoint, method, ctxt, args)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
    result = func(ctxt, **new_args)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 244, in inner
    return func(*args, **kwargs)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 102, in wrapper
    return fn(self, context, *args, **kwargs)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/utils.py", line 1453, in decorated_function
    return function(self, context, *args, **kwargs)

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 327, in migrate_server
    self._live_migrate(context, instance, scheduler_hint,

  File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 534, in _live_migrate
    raise exception.MigrationError(reason=str(ex))

nova.exception.MigrationError: Migration error: rule:create_port_binding is disallowed by policy
Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/api/openstack/wsgi.py", line 658, in wrapped
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   [Previous line repeated 1 more time]
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/api/openstack/compute/migrate_server.py", line 126, in _migrate_live
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self.compute_api.live_migrate(context, instance, block_migration,
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 389, in inner
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return function(self, context, instance, *args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 374, in wrapper
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return func(self, context, instance, *args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 310, in inner
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return f(self, context, instance, *args, **kw)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 357, in wrapper
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return func(self, context, instance, *args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 279, in inner
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return f(self, context, instance, *args, **kw)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 260, in inner
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return f(self, context, instance, *args, **kw)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 242, in inner
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return function(self, context, instance, *args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 168, in inner
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return f(self, context, instance, *args, **kw)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 5576, in live_migrate
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self.compute_task_api.live_migrate_instance(context, instance,
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/api.py", line 110, in live_migrate_instance
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self.conductor_compute_rpcapi.migrate_server(
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/rpcapi.py", line 352, in migrate_server
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return cctxt.call(context, 'migrate_server', **kw)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     result = self.transport._send(
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return self._driver.send(target, ctxt, message,
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return self._send(target, ctxt, message, wait_for_reply, timeout,
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     raise result
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi nova.exception_Remote.MigrationError_Remote: Migration error: rule:create_port_binding is disallowed by policy
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     ret = obj(*args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     exception_handler_v20(status_code, error_body)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     raise client_exc(message=error_message,
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Forbidden: rule:create_port_binding is disallowed by policy
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred:
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 505, in _live_migrate
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     task.execute()
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/base.py", line 25, in wrap
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     with excutils.save_and_reraise_exception():
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self.force_reraise()
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     raise self.value
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/base.py", line 23, in wrap
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return original(self)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/base.py", line 40, in execute
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return self._execute()
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 100, in _execute
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self.destination, dest_node, self.limits = self._find_destination()
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 550, in _find_destination
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self._call_livem_checks_on_host(host, provider_mapping)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 370, in _call_livem_checks_on_host
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     bindings = self._bind_ports_on_destination(
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 411, in _bind_ports_on_destination
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     bindings = self.network_api.bind_ports_to_host(
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 1541, in bind_ports_to_host
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     binding = client.create_port_binding(port_id, data)['binding']
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     ret = obj(*args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 837, in create_port_binding
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return self.post(self.port_bindings_path % port_id, body=body)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     ret = obj(*args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 361, in post
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return self.do_request("POST", action, body=body,
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     ret = obj(*args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 297, in do_request
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self._handle_fault_response(status_code, replybody, resp)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 214, in wrapper
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     raise exception.Forbidden(str(e))
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi nova.exception.Forbidden: rule:create_port_binding is disallowed by policy
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred:
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     res = self.dispatcher.dispatch(message)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return self._do_dispatch(endpoint, method, ctxt, args)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     result = func(ctxt, **new_args)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 244, in inner
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 102, in wrapper
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return fn(self, context, *args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/utils.py", line 1453, in decorated_function
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return function(self, context, *args, **kwargs)
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 327, in migrate_server
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self._live_migrate(context, instance, scheduler_hint,
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 534, in _live_migrate
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     raise exception.MigrationError(reason=str(ex))
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi nova.exception.MigrationError: Migration error: rule:create_port_binding is disallowed by policy
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
2024-02-01 12:19:43.313 1010 INFO nova.api.openstack.wsgi [None req-d5465c38-0025-4c44-ac5b-5261cbe9d469 645810eb66cb45f38220ad1ade8447fb 073b3acb79724ec0a8246c8984489c58 - - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception_Remote.MigrationError_Remote'>
2024-02-01 12:19:43.314 1010 INFO nova.api.openstack.requestlog [None req-d5465c38-0025-4c44-ac5b-5261cbe9d469 645810eb66cb45f38220ad1ade8447fb 073b3acb79724ec0a8246c8984489c58 - - default default] 172.16.1.12 "POST /v2.1/servers/dade6c95-28a5-41bd-a4e8-0524531db5b3/action" status: 500 len: 217 microversion: 2.1 time: 4.294278

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2051907

Title:
  Failed to live migrate instance to another host

Status in OpenStack Compute (nova):
  New

Bug description:
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi [None req-d5465c38-0025-4c44-ac5b-5261cbe9d469 645810eb66cb45f38220ad1ade8447fb 073b3acb79724ec0a8246c8984489c58 - - default default] Unexpected exception in API method: nova.exception_Remote.MigrationError_Remote: Migration error: rule:create_port_binding is disallowed by policy
  Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']
  Traceback (most recent call last):

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
      ret = obj(*args, **kwargs)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response
      exception_handler_v20(status_code, error_body)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20
      raise client_exc(message=error_message,

  neutronclient.common.exceptions.Forbidden: rule:create_port_binding is disallowed by policy
  Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']

  
  During handling of the above exception, another exception occurred:

  
  Traceback (most recent call last):

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 505, in _live_migrate
      task.execute()

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/base.py", line 25, in wrap
      with excutils.save_and_reraise_exception():

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
      self.force_reraise()

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
      raise self.value

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/base.py", line 23, in wrap
      return original(self)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/base.py", line 40, in execute
      return self._execute()

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 100, in _execute
      self.destination, dest_node, self.limits = self._find_destination()

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 550, in _find_destination
      self._call_livem_checks_on_host(host, provider_mapping)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 370, in _call_livem_checks_on_host
      bindings = self._bind_ports_on_destination(

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 411, in _bind_ports_on_destination
      bindings = self.network_api.bind_ports_to_host(

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 1541, in bind_ports_to_host
      binding = client.create_port_binding(port_id, data)['binding']

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
      ret = obj(*args, **kwargs)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 837, in create_port_binding
      return self.post(self.port_bindings_path % port_id, body=body)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
      ret = obj(*args, **kwargs)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 361, in post
      return self.do_request("POST", action, body=body,

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
      ret = obj(*args, **kwargs)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 297, in do_request
      self._handle_fault_response(status_code, replybody, resp)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 214, in wrapper
      raise exception.Forbidden(str(e))

  nova.exception.Forbidden: rule:create_port_binding is disallowed by policy
  Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']

  
  During handling of the above exception, another exception occurred:

  
  Traceback (most recent call last):

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
      res = self.dispatcher.dispatch(message)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
      return self._do_dispatch(endpoint, method, ctxt, args)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
      result = func(ctxt, **new_args)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 244, in inner
      return func(*args, **kwargs)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 102, in wrapper
      return fn(self, context, *args, **kwargs)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/utils.py", line 1453, in decorated_function
      return function(self, context, *args, **kwargs)

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 327, in migrate_server
      self._live_migrate(context, instance, scheduler_hint,

    File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 534, in _live_migrate
      raise exception.MigrationError(reason=str(ex))

  nova.exception.MigrationError: Migration error: rule:create_port_binding is disallowed by policy
  Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/api/openstack/wsgi.py", line 658, in wrapped
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   [Previous line repeated 1 more time]
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/api/openstack/compute/migrate_server.py", line 126, in _migrate_live
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self.compute_api.live_migrate(context, instance, block_migration,
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 389, in inner
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return function(self, context, instance, *args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 374, in wrapper
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return func(self, context, instance, *args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 310, in inner
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return f(self, context, instance, *args, **kw)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 357, in wrapper
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return func(self, context, instance, *args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 279, in inner
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return f(self, context, instance, *args, **kw)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 260, in inner
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return f(self, context, instance, *args, **kw)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 242, in inner
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return function(self, context, instance, *args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 168, in inner
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return f(self, context, instance, *args, **kw)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/api.py", line 5576, in live_migrate
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self.compute_task_api.live_migrate_instance(context, instance,
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/api.py", line 110, in live_migrate_instance
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self.conductor_compute_rpcapi.migrate_server(
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/rpcapi.py", line 352, in migrate_server
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return cctxt.call(context, 'migrate_server', **kw)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     result = self.transport._send(
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return self._driver.send(target, ctxt, message,
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return self._send(target, ctxt, message, wait_for_reply, timeout,
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     raise result
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi nova.exception_Remote.MigrationError_Remote: Migration error: rule:create_port_binding is disallowed by policy
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     ret = obj(*args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     exception_handler_v20(status_code, error_body)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     raise client_exc(message=error_message,
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Forbidden: rule:create_port_binding is disallowed by policy
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred:
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 505, in _live_migrate
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     task.execute()
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/base.py", line 25, in wrap
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     with excutils.save_and_reraise_exception():
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self.force_reraise()
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     raise self.value
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/base.py", line 23, in wrap
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return original(self)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/base.py", line 40, in execute
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return self._execute()
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 100, in _execute
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self.destination, dest_node, self.limits = self._find_destination()
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 550, in _find_destination
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self._call_livem_checks_on_host(host, provider_mapping)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 370, in _call_livem_checks_on_host
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     bindings = self._bind_ports_on_destination(
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/tasks/live_migrate.py", line 411, in _bind_ports_on_destination
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     bindings = self.network_api.bind_ports_to_host(
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 1541, in bind_ports_to_host
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     binding = client.create_port_binding(port_id, data)['binding']
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     ret = obj(*args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 837, in create_port_binding
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return self.post(self.port_bindings_path % port_id, body=body)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     ret = obj(*args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 361, in post
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return self.do_request("POST", action, body=body,
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     ret = obj(*args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 297, in do_request
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self._handle_fault_response(status_code, replybody, resp)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/network/neutron.py", line 214, in wrapper
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     raise exception.Forbidden(str(e))
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi nova.exception.Forbidden: rule:create_port_binding is disallowed by policy
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred:
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     res = self.dispatcher.dispatch(message)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return self._do_dispatch(endpoint, method, ctxt, args)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     result = func(ctxt, **new_args)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 244, in inner
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 102, in wrapper
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return fn(self, context, *args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/compute/utils.py", line 1453, in decorated_function
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     return function(self, context, *args, **kwargs)
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 327, in migrate_server
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     self._live_migrate(context, instance, scheduler_hint,
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi   File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/conductor/manager.py", line 534, in _live_migrate
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi     raise exception.MigrationError(reason=str(ex))
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi nova.exception.MigrationError: Migration error: rule:create_port_binding is disallowed by policy
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3bb8c32a-f305-47b6-91d9-90d7fb8b3523']
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.310 1010 ERROR nova.api.openstack.wsgi 
  2024-02-01 12:19:43.313 1010 INFO nova.api.openstack.wsgi [None req-d5465c38-0025-4c44-ac5b-5261cbe9d469 645810eb66cb45f38220ad1ade8447fb 073b3acb79724ec0a8246c8984489c58 - - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'nova.exception_Remote.MigrationError_Remote'>
  2024-02-01 12:19:43.314 1010 INFO nova.api.openstack.requestlog [None req-d5465c38-0025-4c44-ac5b-5261cbe9d469 645810eb66cb45f38220ad1ade8447fb 073b3acb79724ec0a8246c8984489c58 - - default default] 172.16.1.12 "POST /v2.1/servers/dade6c95-28a5-41bd-a4e8-0524531db5b3/action" status: 500 len: 217 microversion: 2.1 time: 4.294278

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2051907/+subscriptions