yahoo-eng-team team mailing list archive

[Launchpad Bug Tracker <1475411@xxxxxxxxxxxxxxxxxx>]

This bug was fixed in the package nova - 1:2014.1.5-0ubuntu1.5

---------------
nova (1:2014.1.5-0ubuntu1.5) trusty; urgency=medium

  * Fix live migration usage of the wrong connector (LP: #1475411)
    - d/p/Fix-live-migrations-usage-of-the-wrong-connector-inf.patch
  * Fix wrong used ProcessExecutionError exception (LP: #1308839)
    - d/p/Fix-wrong-used-ProcessExecutionError-exception.patch
  * Clean up iSCSI multipath devices in Post Live Migration (LP: #1357368)
    - d/p/Clean-up-iSCSI-multipath-devices-in-Post-Live-Migrat.patch
  * Detach iSCSI latest path for latest disk (LP: #1374999)
    - d/p/Detach-iSCSI-latest-path-for-latest-disk.patch

 -- Billy Olsen <billy.olsen@xxxxxxxxxxxxx>  Fri, 29 Apr 2016 15:35:01
-0700

** Changed in: nova (Ubuntu Trusty)
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1475411

Title:
  During post_live_migration the nova libvirt driver assumes that the
  destination connection info is the same as the source, which is not
  always true

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) juno series:
  Fix Released
Status in OpenStack Compute (nova) kilo series:
  Fix Released
Status in nova package in Ubuntu:
  Fix Released
Status in nova source package in Trusty:
  Fix Released

Bug description:
  [Impact]

  The post_live_migration step for Nova libvirt driver is currently
  making a bad assumption about the source and destination connector
  information. The destination connection info may be different from the
  source which ends up causing LUNs to be left dangling on the source as
  the BDM has overridden the connection info with that of the
  destination.

  Code section where this problem is occuring:

  https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L6036

  At line 6038 the potentially wrong connection info will be passed to
  _disconnect_volume which then ends up not finding the proper LUNs to
  remove (and potentially removes the LUNs for a different volume
  instead).

  By adding debug logging after line 6036 and then comparing that to the
  connection info of the source host (by making a call to Cinder's
  initialize_connection API) you can see that the connection info does
  not match:

  http://paste.openstack.org/show/TjBHyPhidRuLlrxuGktz/

  Version of nova being used:

  commit 35375133398d862a61334783c1e7a90b95f34cdb
  Merge: 83623dd b2c5542
  Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
  Date:   Thu Jul 16 02:01:05 2015 +0000

      Merge "Port crypto to Python 3"

  
  [Test Case]

  Live migrate an instance which is connected to a volume through multi-
  path in which the source and target connection information is not the
  same. Verify that the correct device/LUN is removed (instead of wrong
  one).

  [Regression Potential]

  The regression potential is small as it has run in newer versions of
  nova for awhile now (since Juno, the release immediately following
  Icehouse). If a regression were to occur it would likely prevent a
  live migration from completing (failing in the post processing),
  leaving the instance in an error state. However, it should be migrated
  to the target hypervisor with access to the LUN so it would require
  manual cleanup of the lun at the source hypervisor and a reset of the
  instance state to active.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1475411/+subscriptions