yahoo-eng-team team mailing list archive

[Kashyap Chamarthy <1913716@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

[This bug was originally reported by Lukas Bezdicka when testing Red
Hat's OpenStack (OSP); but this should be reproducible in upstream
context as well.  I'm writing this report based on the root cause
analysis in the environment where the bug occcurred.  Thanks to Daniel
Berrangé for the debugging help!]

Description
-----------

Live-migrating a guest from 'Queens' compute node (running CentOS 7) to
a 'Train' compute node (running CentOS 8) fails with:

-----------------------------------------------------------------------
[...]
 _compare_cpu /usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py:8559
2021-01-26 23:30:25.169 7 ERROR nova.virt.libvirt.driver [req-774be110-7fb6-4865-a177-d624a821cf9e 19ec0130b8714aac8c64a5c2ee5b914b 352675f5f34d45d59bdd61fde58e4bd0 - default default] CPU doesn't have compatibility.

0

Refer to http://libvirt.org/html/libvirt-libvirt-host.html#virCPUCompareResult
2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server [req-774be110-7fb6-4865-a177-d624a821cf9e 19ec0130b8714aac8c64a5c2ee5b914b 352675f5f34d45d59bdd61fde58e4bd0 - default default] Exception during message handling: nova.exception.InvalidCPUInfo: Unacceptable CPU info: CPU doesn't have compatibility.

[...]

2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server     block_migration, disk_over_commit)
2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 8258, in check_can_live_migrate_destination
2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server     self._compare_cpu(None, source_cpu_info, instance)
2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 8575, in _compare_cpu
2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server     raise exception.InvalidCPUInfo(reason=m % {'ret': ret, 'u': u})
2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server nova.exception.InvalidCPUInfo: Unacceptable CPU info: CPU doesn't have compatibility.
2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server
[...]
-----------------------------------------------------------------------

Environment
-----------

The bug was reported by testing in a nested KVM environment, running on
Intel hardware (Xeon(R) Gold 5218R CPU @ 2.10GHz), with the entire
OpenStack setup in VMs.  So the Nova instances themselves will be nested
guests.

  - Source: a CentOS-7 compute node (a level-1 guest), running OpenStack
    'Queens'

  - Destination: a CentOS-8 compute node (a level-1 guest), running
    OpenStack 'Train'

Steps to reproduce
------------------

Live-migrate a guest from source to host.

Expected result
---------------

Live migration should've succeeded.


Actual result
-------------

Live migration fails during compareCPU() check on the destination host
with:

[...]
 _compare_cpu /usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py:8559
2021-01-26 23:30:25.169 7 ERROR nova.virt.libvirt.driver [req-774be110-7fb6-4865-a177-d624a821cf9e 19ec0130b8714aac8c64a5c2ee5b914b 352675f5f34d45d59bdd61fde58e4bd0 - default default] CPU doesn't have compatibility.
[...]

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1913716

Title:
  Live-migrating a guest from 'Queens' to 'Train' fails during libvirt's
  compareCPU() check

Status in OpenStack Compute (nova):
  New

Bug description:
  [This bug was originally reported by Lukas Bezdicka when testing Red
  Hat's OpenStack (OSP); but this should be reproducible in upstream
  context as well.  I'm writing this report based on the root cause
  analysis in the environment where the bug occcurred.  Thanks to Daniel
  Berrangé for the debugging help!]

  Description
  -----------

  Live-migrating a guest from 'Queens' compute node (running CentOS 7) to
  a 'Train' compute node (running CentOS 8) fails with:

  -----------------------------------------------------------------------
  [...]
   _compare_cpu /usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py:8559
  2021-01-26 23:30:25.169 7 ERROR nova.virt.libvirt.driver [req-774be110-7fb6-4865-a177-d624a821cf9e 19ec0130b8714aac8c64a5c2ee5b914b 352675f5f34d45d59bdd61fde58e4bd0 - default default] CPU doesn't have compatibility.

  0

  Refer to http://libvirt.org/html/libvirt-libvirt-host.html#virCPUCompareResult
  2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server [req-774be110-7fb6-4865-a177-d624a821cf9e 19ec0130b8714aac8c64a5c2ee5b914b 352675f5f34d45d59bdd61fde58e4bd0 - default default] Exception during message handling: nova.exception.InvalidCPUInfo: Unacceptable CPU info: CPU doesn't have compatibility.

  [...]

  2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server     block_migration, disk_over_commit)
  2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 8258, in check_can_live_migrate_destination
  2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server     self._compare_cpu(None, source_cpu_info, instance)
  2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 8575, in _compare_cpu
  2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server     raise exception.InvalidCPUInfo(reason=m % {'ret': ret, 'u': u})
  2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server nova.exception.InvalidCPUInfo: Unacceptable CPU info: CPU doesn't have compatibility.
  2021-01-26 23:30:25.242 7 ERROR oslo_messaging.rpc.server
  [...]
  -----------------------------------------------------------------------

  Environment
  -----------

  The bug was reported by testing in a nested KVM environment, running on
  Intel hardware (Xeon(R) Gold 5218R CPU @ 2.10GHz), with the entire
  OpenStack setup in VMs.  So the Nova instances themselves will be nested
  guests.

    - Source: a CentOS-7 compute node (a level-1 guest), running OpenStack
      'Queens'

    - Destination: a CentOS-8 compute node (a level-1 guest), running
      OpenStack 'Train'

  Steps to reproduce
  ------------------

  Live-migrate a guest from source to host.

  Expected result
  ---------------

  Live migration should've succeeded.

  
  Actual result
  -------------

  Live migration fails during compareCPU() check on the destination host
  with:

  [...]
   _compare_cpu /usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py:8559
  2021-01-26 23:30:25.169 7 ERROR nova.virt.libvirt.driver [req-774be110-7fb6-4865-a177-d624a821cf9e 19ec0130b8714aac8c64a5c2ee5b914b 352675f5f34d45d59bdd61fde58e4bd0 - default default] CPU doesn't have compatibility.
  [...]

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1913716/+subscriptions