← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1685340] Re: compute logs tell me live migration finished successfully when it actually failed

 

The actual failure in this case happened on the source host:

http://logs.openstack.org/43/458843/1/check/gate-tempest-dsvm-multinode-
live-migration-ubuntu-
xenial/697a501/logs/subnode-2/libvirt/qemu/instance-00000001.txt.gz

qemu-system-x86_64: /build/qemu-5OJ39u/qemu-2.8+dfsg/block/io.c:1514:
bdrv_co_pwritev: Assertion `!(bs->open_flags & BDRV_O_INACTIVE)' failed.

And shows up like this on the dest host:

http://logs.openstack.org/43/458843/1/check/gate-tempest-dsvm-multinode-
live-migration-ubuntu-
xenial/697a501/logs/libvirt/qemu/instance-00000001.txt.gz

/build/qemu-
5OJ39u/qemu-2.8+dfsg/nbd/server.c:nbd_receive_request():L710: read
failed

This ML post is related:

http://lists.nongnu.org/archive/html/qemu-devel/2017-04/msg01086.html

** Also affects: nova/newton
   Importance: Undecided
       Status: New

** Also affects: nova/ocata
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1685340

Title:
  compute logs tell me live migration finished successfully when it
  actually failed

Status in OpenStack Compute (nova):
  In Progress
Status in OpenStack Compute (nova) newton series:
  New
Status in OpenStack Compute (nova) ocata series:
  New

Bug description:
  This tells me post live migration at destination failed:

  http://logs.openstack.org/43/458843/1/check/gate-tempest-dsvm-
  multinode-live-migration-ubuntu-
  xenial/697a501/logs/subnode-2/screen-n-cpu.txt.gz#_2017-04-21_13_54_10_281

  2017-04-21 13:54:10.281 10362 ERROR nova.compute.manager [req-
  7ecbf938-9e55-4e4c-b7da-63eef0f8d4a9 tempest-
  LiveBlockMigrationTestJSON-208732686 tempest-
  LiveBlockMigrationTestJSON-208732686] [instance: 9bf9f268-5242-4b1d-
  8fe6-ee348b2b8d3e] Post live migration at destination ubuntu-xenial-2
  -node-osic-cloud1-s3500-8527282 failed

  Later on, the logs tell me it was successful:

  http://logs.openstack.org/43/458843/1/check/gate-tempest-dsvm-
  multinode-live-migration-ubuntu-
  xenial/697a501/logs/subnode-2/screen-n-cpu.txt.gz#_2017-04-21_13_54_11_080

  2017-04-21 13:54:11.080 10362 INFO nova.compute.manager [req-
  7ecbf938-9e55-4e4c-b7da-63eef0f8d4a9 tempest-
  LiveBlockMigrationTestJSON-208732686 tempest-
  LiveBlockMigrationTestJSON-208732686] [instance: 9bf9f268-5242-4b1d-
  8fe6-ee348b2b8d3e] Migrating instance to ubuntu-xenial-2-node-osic-
  cloud1-s3500-8527282 finished successfully.

  That's because we don't stop on the failure because we want to
  continue with cleanup, but we don't check if we failed when emitting
  the success message.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1685340/+subscriptions


References