yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #82160
[Bug 1870558] [NEW] Server's host not changed but actually on dest node after live-migrating
Public bug reported:
Description
===========
The insances has been migrating for more than two hours. Then it got error 'Unauthorized'.
The host of output of CLI `openstack server show` was stall the old one.
But the instances had already been running on dest node.
Steps to reproduce
==================
1. Create one instance with large mem
2. Run some application which cosume mem, like `memtester`
3. Execute live-migrate
Expected result
===============
Rollback instance to old one, or update instance's host to dest node
Actual result
=============
Instance on dest node but the host is src node in DB
Environment
===========
$ git log -1
commit ee6af34437069a23284f4521330057a95f86f9b7 (HEAD -> stable/rocky, origin/stable/rocky)
Author: Luigi Toscano <ltoscano@xxxxxxxxxx>
Date: Wed Dec 18 00:28:15 2019 +0100
Zuul v3: use devstack-plugin-nfs-tempest-full
... and replace its legacy ancestor.
Change-Id: Ifd4387a02b3103e1258e146e63c73be1ad10030c
(cherry picked from commit e7e39b8c2e20f5d7b5e70020f0e42541dc772e68)
(cherry picked from commit e82e1704caa1c2baea29f05e8d426337e8de7a3c)
(cherry picked from commit 99aa8ebc12949f9bba76f22e877b07d02791bf5b)
Logs & Configs
==============
2020-04-02 21:08:32,890.890 6358 INFO nova.virt.libvirt.driver [req-b8d694f5-f60a-4866-bcd2-c107b2caa809 bdb83637364c4db4ba1a01f6ea879ff1 496db91424
254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Migration running for 30 secs, memory 80% remaining; (byt
es processed=3503551373, remaining=27653689344, total=34364792832)
2020-04-02 23:08:05,165.165 6358 INFO nova.virt.libvirt.driver [req-f22d9bee-9c1f-47a6-a2d5-3611f5b2529c bdb83637364c4db4ba1a01f6ea879ff1 496db91424254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Migration operation has completed
2020-04-02 23:08:05,166.166 6358 INFO nova.compute.manager [req-f22d9bee-9c1f-47a6-a2d5-3611f5b2529c bdb83637364c4db4ba1a01f6ea879ff1 496db91424254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] _post_live_migration() is started..
2020-04-02 23:08:05,535.535 6358 WARNING nova.virt.libvirt.driver [req-f22d9bee-9c1f-47a6-a2d5-3611f5b2529c bdb83637364c4db4ba1a01f6ea879ff1 496db91424254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Error monitoring migration: The request you have made requires authentication. (HTTP 401): Unauthorized: The request you have made requires authentication. (HTTP 401)
2020-04-02 23:08:05,537.537 6358 ERROR nova.compute.manager [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Unauthorized: The request you have made requires authentication. (HTTP 401)
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1870558
Title:
Server's host not changed but actually on dest node after live-
migrating
Status in OpenStack Compute (nova):
New
Bug description:
Description
===========
The insances has been migrating for more than two hours. Then it got error 'Unauthorized'.
The host of output of CLI `openstack server show` was stall the old one.
But the instances had already been running on dest node.
Steps to reproduce
==================
1. Create one instance with large mem
2. Run some application which cosume mem, like `memtester`
3. Execute live-migrate
Expected result
===============
Rollback instance to old one, or update instance's host to dest node
Actual result
=============
Instance on dest node but the host is src node in DB
Environment
===========
$ git log -1
commit ee6af34437069a23284f4521330057a95f86f9b7 (HEAD -> stable/rocky, origin/stable/rocky)
Author: Luigi Toscano <ltoscano@xxxxxxxxxx>
Date: Wed Dec 18 00:28:15 2019 +0100
Zuul v3: use devstack-plugin-nfs-tempest-full
... and replace its legacy ancestor.
Change-Id: Ifd4387a02b3103e1258e146e63c73be1ad10030c
(cherry picked from commit e7e39b8c2e20f5d7b5e70020f0e42541dc772e68)
(cherry picked from commit e82e1704caa1c2baea29f05e8d426337e8de7a3c)
(cherry picked from commit 99aa8ebc12949f9bba76f22e877b07d02791bf5b)
Logs & Configs
==============
2020-04-02 21:08:32,890.890 6358 INFO nova.virt.libvirt.driver [req-b8d694f5-f60a-4866-bcd2-c107b2caa809 bdb83637364c4db4ba1a01f6ea879ff1 496db91424
254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Migration running for 30 secs, memory 80% remaining; (byt
es processed=3503551373, remaining=27653689344, total=34364792832)
2020-04-02 23:08:05,165.165 6358 INFO nova.virt.libvirt.driver [req-f22d9bee-9c1f-47a6-a2d5-3611f5b2529c bdb83637364c4db4ba1a01f6ea879ff1 496db91424254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Migration operation has completed
2020-04-02 23:08:05,166.166 6358 INFO nova.compute.manager [req-f22d9bee-9c1f-47a6-a2d5-3611f5b2529c bdb83637364c4db4ba1a01f6ea879ff1 496db91424254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] _post_live_migration() is started..
2020-04-02 23:08:05,535.535 6358 WARNING nova.virt.libvirt.driver [req-f22d9bee-9c1f-47a6-a2d5-3611f5b2529c bdb83637364c4db4ba1a01f6ea879ff1 496db91424254a85a4130a26801447c9 - default default] [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Error monitoring migration: The request you have made requires authentication. (HTTP 401): Unauthorized: The request you have made requires authentication. (HTTP 401)
2020-04-02 23:08:05,537.537 6358 ERROR nova.compute.manager [instance: 8e76d7a1-e7f4-4476-94b3-724db6bfd467] Unauthorized: The request you have made requires authentication. (HTTP 401)
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1870558/+subscriptions
Follow ups