| Thread Previous • Date Previous • Date Next • Thread Next |
Public bug reported:
Brief Description
-----------------
Live migration succeeds but instance-action-list still has unexpected Error status
Severity
--------
standard
Steps to Reproduce
------------------
1. Launch instance from glance image with flavor
for eg. 3vcpu, 1GB ram, 14GB disk, hw:cpu_policy, hw:mem_page_size=2048
ID b457c0d6-ee31-43da-a481-bc3ad5663d7c
*If it scheduled on compute-0, migrate it away from compute-0 (cold
migrate) to one of the other computes eg. compute-2
3. Attempt to live (block) migration away from compute-2
The instance schedules and appears to have completed on compute-1
Expected Behavior
------------------
Expect the instance to autoschedule (live migrate) on the compatible host
Did not expect nova instance-action-list to report Error (at least when the instance was autoscheduled and succeeded the migration)
Actual Behavior
----------------
Error is reported in the nova instance-action-list (even though it autoscheduled and moved to the host)
$ nova instance-action-list b457c0d6-ee31-43da-a481-bc3ad5663d7c
Action | Request_ID | Message | Start_Time |
Updated_At |
...
| live-migration | req-a9598f9c-0fab-4a2d-afbe-7d362ea9bb64 | Error | 2019-04-11T20:24:49.000000 | 2019-04-11T20:25:09.000000
$ nova instance-action b457c0d6-ee31-43da-a481-bc3ad5663d7c req-a9598f9c-0fab-4a2d-afbe-7d362ea9bb64
| Property | Value
| action | live-migration |
| events | [{u'event': u'compute_post_live_migration_at_destination', |
| | u'finish_time': u'2019-04-11T20:25:09.000000', |
| | u'host': u'compute-1', |
| | u'hostId': u'5420963ef16628588a36ecea76a71540080b5f7a01d77b37b276aad6', |
| | u'result': u'Success', |
| | u'start_time': u'2019-04-11T20:25:07.000000', |
| | u'traceback': None}, |
| | {u'event': u'compute_pre_live_migration', |
| | u'finish_time': u'2019-04-11T20:24:58.000000', |
| | u'host': u'compute-1', |
| | u'hostId': u'5420963ef16628588a36ecea76a71540080b5f7a01d77b37b276aad6', |
| | u'result': u'Success', |
| | u'start_time': u'2019-04-11T20:24:53.000000', |
| | u'traceback': None}, |
| | {u'event': u'compute_live_migration', |
| | u'finish_time': u'2019-04-11T20:24:53.000000', |
| | u'host': u'compute-2', |
| | u'hostId': u'cf35d970d7214ea427046101a98b0c29734e063dd82f9ac668bf061c', |
| | u'result': u'Success', |
| | u'start_time': u'2019-04-11T20:24:53.000000', |
| | u'traceback': None}, |
| | {u'event': u'compute_check_can_live_migrate_source', |
| | u'finish_time': u'2019-04-11T20:24:52.000000', |
| | u'host': u'compute-2', |
| | u'hostId': u'cf35d970d7214ea427046101a98b0c29734e063dd82f9ac668bf061c', |
| | u'result': u'Success', |
| | u'start_time': u'2019-04-11T20:24:52.000000', |
| | u'traceback': None}, |
| | {u'event': u'compute_check_can_live_migrate_destination', |
| | u'finish_time': None, |
| | u'host': u'compute-1', |
| | u'hostId': u'5420963ef16628588a36ecea76a71540080b5f7a01d77b37b276aad6', |
| | u'result': None, |
| | u'start_time': u'2019-04-11T20:24:52.000000', |
| | u'traceback': None}, |
| | {u'event': u'compute_check_can_live_migrate_destination', |
| | u'finish_time': u'2019-04-11T20:24:52.000000', |
| | u'host': u'compute-0', |
| | u'hostId': u'4ab1941a198403c0b8b1a26c1f9e39536fdd884b4c4d1a8ab381a30b', |
| | u'result': u'Success', |
| | u'start_time': u'2019-04-11T20:24:51.000000', |
| | u'traceback': u' File "/var/lib/openstack/lib/python2.7/site-packages/nova/compute/utils.py", line 1323, in decorated_function |
| | return function(self, context, *args, **kwargs) |
| | File "/var/lib/openstack/lib/python2.7/site-packages/nova/compute/manager.py", line 214, in decorated_function |
| | kwargs[\'instance\'], e, sys.exc_info()) |
| | File "/var/lib/openstack/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ |
| | self.force_reraise() |
| | File "/var/lib/openstack/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise |
| | six.reraise(self.type_, self.value, self.tb) |
| | File "/var/lib/openstack/lib/python2.7/site-packages/nova/compute/manager.py", line 202, in decorated_function |
| | return function(self, context, *args, **kwargs) |
| | File "/var/lib/openstack/lib/python2.7/site-packages/nova/compute/manager.py", line 6203, in check_can_live_migrate_destination |
| | block_migration, disk_over_commit) |
| | File "/var/lib/openstack/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7195, in check_can_live_migrate_destination |
| | self._compare_cpu(None, source_cpu_info, instance) |
| | File "/var/lib/openstack/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7446, in _compare_cpu |
| | raise exception.InvalidCPUInfo(reason=m % {\'ret\': ret, \'u\': u}) |
| | '}, |
| | {u'event': u'conductor_migrate_server', |
| | u'finish_time': u'2019-04-11T20:24:53.000000', |
| | u'host': u'nova-conductor-6f96f9bf6f-z7xqm', |
| | u'hostId': u'b94f767d471ef257391e8d24d33e3fee0c7b37a6f3c9871d6b6ebe00', |
| | u'result': u'Success', |
| | u'start_time': u'2019-04-11T20:24:49.000000', |
| | u'traceback': None}] |
| instance_uuid | b457c0d6-ee31-43da-a481-bc3ad5663d7c |
| message | Error |
| project_id | 70c3556265ee4617981d506b0f5bbfeb |
| request_id | req-a9598f9c-0fab-4a2d-afbe-7d362ea9bb64 |
| start_time | 2019-04-11T20:24:49.000000 |
| updated_at | 2019-04-11T20:25:09.000000 |
| user_id | 319e82fc9a89424a92787ab4d91d0161 |
+---------------+------------------------------------
Reproducibility
---------------
yes
System Configuration
--------------------
2+3
(Lab yow-cgcs-wildcat-92_98)
compute-0:
Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz
Applications
Processor 0 : 3-21,47-65
Processor 1 : 22-43,66-87
compute-1 and compute-2:
Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz
Applications Processor 0 : 3-17,39-53
Processor 1 : 18-35,54-71
Branch/Pull Time/Commit
--------------------
BUILD_ID="20190410T013000Z"
JOB="STX_build_master_master"
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1824420
Title:
Live migration succeeds but instance-action-list still has unexpected
Error status
Status in OpenStack Compute (nova):
New
Bug description:
Brief Description
-----------------
Live migration succeeds but instance-action-list still has unexpected Error status
Severity
--------
standard
Steps to Reproduce
------------------
1. Launch instance from glance image with flavor
for eg. 3vcpu, 1GB ram, 14GB disk, hw:cpu_policy, hw:mem_page_size=2048
ID b457c0d6-ee31-43da-a481-bc3ad5663d7c
*If it scheduled on compute-0, migrate it away from compute-0 (cold
migrate) to one of the other computes eg. compute-2
3. Attempt to live (block) migration away from compute-2
The instance schedules and appears to have completed on compute-1
Expected Behavior
------------------
Expect the instance to autoschedule (live migrate) on the compatible host
Did not expect nova instance-action-list to report Error (at least when the instance was autoscheduled and succeeded the migration)
Actual Behavior
----------------
Error is reported in the nova instance-action-list (even though it autoscheduled and moved to the host)
$ nova instance-action-list b457c0d6-ee31-43da-a481-bc3ad5663d7c
Action | Request_ID | Message | Start_Time |
Updated_At |
...
| live-migration | req-a9598f9c-0fab-4a2d-afbe-7d362ea9bb64 | Error | 2019-04-11T20:24:49.000000 | 2019-04-11T20:25:09.000000
$ nova instance-action b457c0d6-ee31-43da-a481-bc3ad5663d7c req-a9598f9c-0fab-4a2d-afbe-7d362ea9bb64
| Property | Value
| action | live-migration |
| events | [{u'event': u'compute_post_live_migration_at_destination', |
| | u'finish_time': u'2019-04-11T20:25:09.000000', |
| | u'host': u'compute-1', |
| | u'hostId': u'5420963ef16628588a36ecea76a71540080b5f7a01d77b37b276aad6', |
| | u'result': u'Success', |
| | u'start_time': u'2019-04-11T20:25:07.000000', |
| | u'traceback': None}, |
| | {u'event': u'compute_pre_live_migration', |
| | u'finish_time': u'2019-04-11T20:24:58.000000', |
| | u'host': u'compute-1', |
| | u'hostId': u'5420963ef16628588a36ecea76a71540080b5f7a01d77b37b276aad6', |
| | u'result': u'Success', |
| | u'start_time': u'2019-04-11T20:24:53.000000', |
| | u'traceback': None}, |
| | {u'event': u'compute_live_migration', |
| | u'finish_time': u'2019-04-11T20:24:53.000000', |
| | u'host': u'compute-2', |
| | u'hostId': u'cf35d970d7214ea427046101a98b0c29734e063dd82f9ac668bf061c', |
| | u'result': u'Success', |
| | u'start_time': u'2019-04-11T20:24:53.000000', |
| | u'traceback': None}, |
| | {u'event': u'compute_check_can_live_migrate_source', |
| | u'finish_time': u'2019-04-11T20:24:52.000000', |
| | u'host': u'compute-2', |
| | u'hostId': u'cf35d970d7214ea427046101a98b0c29734e063dd82f9ac668bf061c', |
| | u'result': u'Success', |
| | u'start_time': u'2019-04-11T20:24:52.000000', |
| | u'traceback': None}, |
| | {u'event': u'compute_check_can_live_migrate_destination', |
| | u'finish_time': None, |
| | u'host': u'compute-1', |
| | u'hostId': u'5420963ef16628588a36ecea76a71540080b5f7a01d77b37b276aad6', |
| | u'result': None, |
| | u'start_time': u'2019-04-11T20:24:52.000000', |
| | u'traceback': None}, |
| | {u'event': u'compute_check_can_live_migrate_destination', |
| | u'finish_time': u'2019-04-11T20:24:52.000000', |
| | u'host': u'compute-0', |
| | u'hostId': u'4ab1941a198403c0b8b1a26c1f9e39536fdd884b4c4d1a8ab381a30b', |
| | u'result': u'Success', |
| | u'start_time': u'2019-04-11T20:24:51.000000', |
| | u'traceback': u' File "/var/lib/openstack/lib/python2.7/site-packages/nova/compute/utils.py", line 1323, in decorated_function |
| | return function(self, context, *args, **kwargs) |
| | File "/var/lib/openstack/lib/python2.7/site-packages/nova/compute/manager.py", line 214, in decorated_function |
| | kwargs[\'instance\'], e, sys.exc_info()) |
| | File "/var/lib/openstack/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ |
| | self.force_reraise() |
| | File "/var/lib/openstack/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise |
| | six.reraise(self.type_, self.value, self.tb) |
| | File "/var/lib/openstack/lib/python2.7/site-packages/nova/compute/manager.py", line 202, in decorated_function |
| | return function(self, context, *args, **kwargs) |
| | File "/var/lib/openstack/lib/python2.7/site-packages/nova/compute/manager.py", line 6203, in check_can_live_migrate_destination |
| | block_migration, disk_over_commit) |
| | File "/var/lib/openstack/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7195, in check_can_live_migrate_destination |
| | self._compare_cpu(None, source_cpu_info, instance) |
| | File "/var/lib/openstack/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7446, in _compare_cpu |
| | raise exception.InvalidCPUInfo(reason=m % {\'ret\': ret, \'u\': u}) |
| | '}, |
| | {u'event': u'conductor_migrate_server', |
| | u'finish_time': u'2019-04-11T20:24:53.000000', |
| | u'host': u'nova-conductor-6f96f9bf6f-z7xqm', |
| | u'hostId': u'b94f767d471ef257391e8d24d33e3fee0c7b37a6f3c9871d6b6ebe00', |
| | u'result': u'Success', |
| | u'start_time': u'2019-04-11T20:24:49.000000', |
| | u'traceback': None}] |
| instance_uuid | b457c0d6-ee31-43da-a481-bc3ad5663d7c |
| message | Error |
| project_id | 70c3556265ee4617981d506b0f5bbfeb |
| request_id | req-a9598f9c-0fab-4a2d-afbe-7d362ea9bb64 |
| start_time | 2019-04-11T20:24:49.000000 |
| updated_at | 2019-04-11T20:25:09.000000 |
| user_id | 319e82fc9a89424a92787ab4d91d0161 |
+---------------+------------------------------------
Reproducibility
---------------
yes
System Configuration
--------------------
2+3
(Lab yow-cgcs-wildcat-92_98)
compute-0:
Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz
Applications
Processor 0 : 3-21,47-65
Processor 1 : 22-43,66-87
compute-1 and compute-2:
Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz
Applications Processor 0 : 3-17,39-53
Processor 1 : 18-35,54-71
Branch/Pull Time/Commit
--------------------
BUILD_ID="20190410T013000Z"
JOB="STX_build_master_master"
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1824420/+subscriptions
| Thread Previous • Date Previous • Date Next • Thread Next |
