yahoo-eng-team team mailing list archive

[Eric Xie <1870558@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Description
===========
The insances has been migrating for more than two hours. Then it got error 'Unauthorized'.
The host of output of CLI `openstack server show` was stall the old one.
But the instances had already been running on dest node.

Steps to reproduce
==================
1. Create one instance with large mem
2. Run some application which cosume mem, like `memtester`
3. Execute live-migrate

Expected result
===============
Rollback instance to old one, or update instance's host to dest node

Actual result
=============
Instance on dest node but the host is src node in DB

Environment
===========
$ git log -1
commit ee6af34437069a23284f4521330057a95f86f9b7 (HEAD -> stable/rocky, origin/stable/rocky)
Author: Luigi Toscano <ltoscano@xxxxxxxxxx>
Date:   Wed Dec 18 00:28:15 2019 +0100

    Zuul v3: use devstack-plugin-nfs-tempest-full

    ... and replace its legacy ancestor.

    Change-Id: Ifd4387a02b3103e1258e146e63c73be1ad10030c
    (cherry picked from commit e7e39b8c2e20f5d7b5e70020f0e42541dc772e68)
    (cherry picked from commit e82e1704caa1c2baea29f05e8d426337e8de7a3c)
    (cherry picked from commit 99aa8ebc12949f9bba76f22e877b07d02791bf5b)

Logs & Configs
==============
2020-04-02 21:08:32,890.890 6358 INFO nova.virt.libvirt.driver [req-b8d694f5-f60a-4866-bcd2-c107b2caa809 bdb83637364c4db4ba1a01f6ea879ff1 496db91424
254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Migration running for 30 secs, memory 80% remaining; (byt
es processed=3503551373, remaining=27653689344, total=34364792832)
2020-04-02 23:08:05,165.165 6358 INFO nova.virt.libvirt.driver [req-f22d9bee-9c1f-47a6-a2d5-3611f5b2529c bdb83637364c4db4ba1a01f6ea879ff1 496db91424254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Migration operation has completed
2020-04-02 23:08:05,166.166 6358 INFO nova.compute.manager [req-f22d9bee-9c1f-47a6-a2d5-3611f5b2529c bdb83637364c4db4ba1a01f6ea879ff1 496db91424254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] _post_live_migration() is started..
2020-04-02 23:08:05,535.535 6358 WARNING nova.virt.libvirt.driver [req-f22d9bee-9c1f-47a6-a2d5-3611f5b2529c bdb83637364c4db4ba1a01f6ea879ff1 496db91424254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Error monitoring migration: The request you have made requires authentication. (HTTP 401): Unauthorized: The request you have made requires authentication. (HTTP 401)
2020-04-02 23:08:05,537.537 6358 ERROR nova.compute.manager [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Unauthorized: The request you have made requires authentication. (HTTP 401)

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1870558

Title:
  Server's host not changed but actually on dest node after live-
  migrating

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========
  The insances has been migrating for more than two hours. Then it got error 'Unauthorized'.
  The host of output of CLI `openstack server show` was stall the old one.
  But the instances had already been running on dest node.

  Steps to reproduce
  ==================
  1. Create one instance with large mem
  2. Run some application which cosume mem, like `memtester`
  3. Execute live-migrate

  Expected result
  ===============
  Rollback instance to old one, or update instance's host to dest node

  Actual result
  =============
  Instance on dest node but the host is src node in DB

  Environment
  ===========
  $ git log -1
  commit ee6af34437069a23284f4521330057a95f86f9b7 (HEAD -> stable/rocky, origin/stable/rocky)
  Author: Luigi Toscano <ltoscano@xxxxxxxxxx>
  Date:   Wed Dec 18 00:28:15 2019 +0100

      Zuul v3: use devstack-plugin-nfs-tempest-full

      ... and replace its legacy ancestor.

      Change-Id: Ifd4387a02b3103e1258e146e63c73be1ad10030c
      (cherry picked from commit e7e39b8c2e20f5d7b5e70020f0e42541dc772e68)
      (cherry picked from commit e82e1704caa1c2baea29f05e8d426337e8de7a3c)
      (cherry picked from commit 99aa8ebc12949f9bba76f22e877b07d02791bf5b)

  Logs & Configs
  ==============
  2020-04-02 21:08:32,890.890 6358 INFO nova.virt.libvirt.driver [req-b8d694f5-f60a-4866-bcd2-c107b2caa809 bdb83637364c4db4ba1a01f6ea879ff1 496db91424
  254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Migration running for 30 secs, memory 80% remaining; (byt
  es processed=3503551373, remaining=27653689344, total=34364792832)
  2020-04-02 23:08:05,165.165 6358 INFO nova.virt.libvirt.driver [req-f22d9bee-9c1f-47a6-a2d5-3611f5b2529c bdb83637364c4db4ba1a01f6ea879ff1 496db91424254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Migration operation has completed
  2020-04-02 23:08:05,166.166 6358 INFO nova.compute.manager [req-f22d9bee-9c1f-47a6-a2d5-3611f5b2529c bdb83637364c4db4ba1a01f6ea879ff1 496db91424254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] _post_live_migration() is started..
  2020-04-02 23:08:05,535.535 6358 WARNING nova.virt.libvirt.driver [req-f22d9bee-9c1f-47a6-a2d5-3611f5b2529c bdb83637364c4db4ba1a01f6ea879ff1 496db91424254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Error monitoring migration: The request you have made requires authentication. (HTTP 401): Unauthorized: The request you have made requires authentication. (HTTP 401)
  2020-04-02 23:08:05,537.537 6358 ERROR nova.compute.manager [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Unauthorized: The request you have made requires authentication. (HTTP 401)

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1870558/+subscriptions