← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1950138] [NEW] instance action event result inconsistent

 

Public bug reported:

Live migration will repeat execute check_can_live_migrate_destination when it Throw exceptions in selected destination. Until check the destination is possible to execute live migration.
The action events will record the result of each execution. But the event result is inconsistent with the actual execution results.

Like this:
The first compute_check_can_live_migrate_destination execution results is Error, but it record Success, the latest execution results is Success, but it record None.

+---------------+-------------------------------------------------------------------------------------------------------------------------+
| Property      | Value                                                                                                                   |
+---------------+-------------------------------------------------------------------------------------------------------------------------+
| action        | live-migration                                                                                                          |
| events        | [{u'event': u'compute_post_live_migration_at_destination',                                                              |
|               |   u'finish_time': u'2021-11-08T08:42:01.000000',                                                                        |
|               |   u'result': u'Success',                                                                                                |
|               |   u'start_time': u'2021-11-08T08:41:58.000000',                                                                         |
|               |   u'traceback': None},                                                                                                  |
|               |  {u'event': u'compute_pre_live_migration',                                                                              |
|               |   u'finish_time': u'2021-11-08T08:41:54.000000',                                                                        |
|               |   u'result': u'Success',                                                                                                |
|               |   u'start_time': u'2021-11-08T08:41:51.000000',                                                                         |
|               |   u'traceback': None},                                                                                                  |
|               |  {u'event': u'compute_live_migration',                                                                                  |
|               |   u'finish_time': u'2021-11-08T08:41:51.000000',                                                                        |
|               |   u'result': u'Success',                                                                                                |
|               |   u'start_time': u'2021-11-08T08:41:51.000000',                                                                         |
|               |   u'traceback': None},                                                                                                  |
|               |  {u'event': u'compute_check_can_live_migrate_source',                                                                   |
|               |   u'finish_time': u'2021-11-08T08:41:51.000000',                                                                        |
|               |   u'result': u'Success',                                                                                                |
|               |   u'start_time': u'2021-11-08T08:41:51.000000',                                                                         |
|               |   u'traceback': None},                                                                                                  |
|               |  {u'event': u'compute_check_can_live_migrate_destination',                                                              |
|               |   u'finish_time': None,                                                                                                 |
|               |   u'result': None,                                                                                                      |
|               |   u'start_time': u'2021-11-08T08:41:51.000000',                                                                         |
|               |   u'traceback': None},                                                                                                  |
|               |  {u'event': u'compute_check_can_live_migrate_destination',                                                              |
|               |   u'finish_time': None,                                                                                                 |
|               |   u'result': None,                                                                                                      |
|               |   u'start_time': u'2021-11-08T08:41:49.000000',                                                                         |
|               |   u'traceback': None},                                                                                                  |
|               |  {u'event': u'compute_check_can_live_migrate_destination',                                                              |
|               |   u'finish_time': u'2021-11-08T08:41:51.000000',                                                                        |
|               |   u'result': u'Success',                                                                                                |
|               |   u'start_time': u'2021-11-08T08:41:47.000000',                                                                         |
|               |   u'traceback': u'  File "/usr/lib/python2.7/site-packages/nova/compute/utils.py", line 976, in decorated_function      |
|               |     return function(self, context, *args, **kwargs)                                                                     |
|               |   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 222, in decorated_function                      |
|               |     kwargs[\'instance\'], e, sys.exc_info())                                                                            |
|               |   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__                                 |
|               |     self.force_reraise()                                                                                                |
|               |   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise                            |
|               |     six.reraise(self.type_, self.value, self.tb)                                                                        |
|               |   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 210, in decorated_function                      |
|               |     return function(self, context, *args, **kwargs)                                                                     |
|               |   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6499, in check_can_live_migrate_destination     |
|               |     disk_over_commit)                                                                                                   |
|               |   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6510, in _do_check_can_live_migrate_destination |
|               |     block_migration, disk_over_commit)                                                                                  |
|               |   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7052, in check_can_live_migrate_destination |
|               |     raise exception.InvalidCPUInfo(reason="Alex test")                                                                  |
|               | '},                                                                                                                     |
|               |  {u'event': u'conductor_live_migrate_instance',                                                                         |
|               |   u'finish_time': u'2021-11-08T08:41:51.000000',                                                                        |
|               |   u'result': u'Success',                                                                                                |
|               |   u'start_time': u'2021-11-08T08:41:44.000000',                                                                         |
|               |   u'traceback': None}]                                                                                                  |
| instance_uuid | eabdfcdc-f96f-4339-9dac-4e4cf77cd047                                                                                    |
| message       | Error                                                                                                                   |
| project_id    | 301c71abbc4443699475be768334962c                                                                                        |
| request_id    | req-35c48cf0-1a4f-4962-80fb-4aa5b726dee9                                                                                |
| start_time    | 2021-11-08T08:41:44.000000                                                                                              |
| updated_at    | 2021-11-08T08:42:01.000000                                                                                              |
| user_id       | 1d0ce6d5cf9643d5825151a87a48cebb                                                                                        |
+---------------+-------------------------------------------------------------------------------------------------------------------------+

** Affects: nova
     Importance: Undecided
     Assignee: wujian (wujian666)
         Status: In Progress

** Changed in: nova
     Assignee: (unassigned) => wujian (wujian666)

** Changed in: nova
       Status: New => In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1950138

Title:
  instance action event result inconsistent

Status in OpenStack Compute (nova):
  In Progress

Bug description:
  Live migration will repeat execute check_can_live_migrate_destination when it Throw exceptions in selected destination. Until check the destination is possible to execute live migration.
  The action events will record the result of each execution. But the event result is inconsistent with the actual execution results.

  Like this:
  The first compute_check_can_live_migrate_destination execution results is Error, but it record Success, the latest execution results is Success, but it record None.

  +---------------+-------------------------------------------------------------------------------------------------------------------------+
  | Property      | Value                                                                                                                   |
  +---------------+-------------------------------------------------------------------------------------------------------------------------+
  | action        | live-migration                                                                                                          |
  | events        | [{u'event': u'compute_post_live_migration_at_destination',                                                              |
  |               |   u'finish_time': u'2021-11-08T08:42:01.000000',                                                                        |
  |               |   u'result': u'Success',                                                                                                |
  |               |   u'start_time': u'2021-11-08T08:41:58.000000',                                                                         |
  |               |   u'traceback': None},                                                                                                  |
  |               |  {u'event': u'compute_pre_live_migration',                                                                              |
  |               |   u'finish_time': u'2021-11-08T08:41:54.000000',                                                                        |
  |               |   u'result': u'Success',                                                                                                |
  |               |   u'start_time': u'2021-11-08T08:41:51.000000',                                                                         |
  |               |   u'traceback': None},                                                                                                  |
  |               |  {u'event': u'compute_live_migration',                                                                                  |
  |               |   u'finish_time': u'2021-11-08T08:41:51.000000',                                                                        |
  |               |   u'result': u'Success',                                                                                                |
  |               |   u'start_time': u'2021-11-08T08:41:51.000000',                                                                         |
  |               |   u'traceback': None},                                                                                                  |
  |               |  {u'event': u'compute_check_can_live_migrate_source',                                                                   |
  |               |   u'finish_time': u'2021-11-08T08:41:51.000000',                                                                        |
  |               |   u'result': u'Success',                                                                                                |
  |               |   u'start_time': u'2021-11-08T08:41:51.000000',                                                                         |
  |               |   u'traceback': None},                                                                                                  |
  |               |  {u'event': u'compute_check_can_live_migrate_destination',                                                              |
  |               |   u'finish_time': None,                                                                                                 |
  |               |   u'result': None,                                                                                                      |
  |               |   u'start_time': u'2021-11-08T08:41:51.000000',                                                                         |
  |               |   u'traceback': None},                                                                                                  |
  |               |  {u'event': u'compute_check_can_live_migrate_destination',                                                              |
  |               |   u'finish_time': None,                                                                                                 |
  |               |   u'result': None,                                                                                                      |
  |               |   u'start_time': u'2021-11-08T08:41:49.000000',                                                                         |
  |               |   u'traceback': None},                                                                                                  |
  |               |  {u'event': u'compute_check_can_live_migrate_destination',                                                              |
  |               |   u'finish_time': u'2021-11-08T08:41:51.000000',                                                                        |
  |               |   u'result': u'Success',                                                                                                |
  |               |   u'start_time': u'2021-11-08T08:41:47.000000',                                                                         |
  |               |   u'traceback': u'  File "/usr/lib/python2.7/site-packages/nova/compute/utils.py", line 976, in decorated_function      |
  |               |     return function(self, context, *args, **kwargs)                                                                     |
  |               |   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 222, in decorated_function                      |
  |               |     kwargs[\'instance\'], e, sys.exc_info())                                                                            |
  |               |   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__                                 |
  |               |     self.force_reraise()                                                                                                |
  |               |   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise                            |
  |               |     six.reraise(self.type_, self.value, self.tb)                                                                        |
  |               |   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 210, in decorated_function                      |
  |               |     return function(self, context, *args, **kwargs)                                                                     |
  |               |   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6499, in check_can_live_migrate_destination     |
  |               |     disk_over_commit)                                                                                                   |
  |               |   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6510, in _do_check_can_live_migrate_destination |
  |               |     block_migration, disk_over_commit)                                                                                  |
  |               |   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7052, in check_can_live_migrate_destination |
  |               |     raise exception.InvalidCPUInfo(reason="Alex test")                                                                  |
  |               | '},                                                                                                                     |
  |               |  {u'event': u'conductor_live_migrate_instance',                                                                         |
  |               |   u'finish_time': u'2021-11-08T08:41:51.000000',                                                                        |
  |               |   u'result': u'Success',                                                                                                |
  |               |   u'start_time': u'2021-11-08T08:41:44.000000',                                                                         |
  |               |   u'traceback': None}]                                                                                                  |
  | instance_uuid | eabdfcdc-f96f-4339-9dac-4e4cf77cd047                                                                                    |
  | message       | Error                                                                                                                   |
  | project_id    | 301c71abbc4443699475be768334962c                                                                                        |
  | request_id    | req-35c48cf0-1a4f-4962-80fb-4aa5b726dee9                                                                                |
  | start_time    | 2021-11-08T08:41:44.000000                                                                                              |
  | updated_at    | 2021-11-08T08:42:01.000000                                                                                              |
  | user_id       | 1d0ce6d5cf9643d5825151a87a48cebb                                                                                        |
  +---------------+-------------------------------------------------------------------------------------------------------------------------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1950138/+subscriptions