yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1708572] Re: Unable to live-migrate : Disk of instance is too large

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: OpenStack Infra <1708572@xxxxxxxxxxxxxxxxxx>
Date: Thu, 14 Dec 2017 17:17:31 -0000
Reply-to: Bug 1708572 <1708572@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Reviewed:  https://review.openstack.org/524681
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e097c001c8e11110efe8879da57264fcb7bdfdf2
Submitter: Zuul
Branch:    master

commit e097c001c8e11110efe8879da57264fcb7bdfdf2
Author: Matthew Booth <mbooth@xxxxxxxxxx>
Date:   Fri Dec 1 16:39:49 2017 +0000

    Fix disk size during live migration with disk over-commit
    
    Prior to microversion 2.25, the migration api supported a
    'disk_over_commit' parameter, which indicated that we should do disk
    size check on the destination host taking into account disk
    over-commit. Versions since 2.25 no longer have this parameter, but we
    continue to support the older microversion.
    
    This disk size check was broken when disk over commit was in use on
    the host. In LibvirtDriver._assert_dest_node_has_enough_disk() we
    correctly calculate the required space using allocated disk size
    rather than virtual disk size when doing an over-committed check.
    However, we were checking against 'disk_available_least' as reported
    by the destination host. This value is: the amount of disk space which
    the host would have if all instances fully allocated their storage. On
    an over-committed host it will therefore be negative, despite there
    actually being space on the destination host.
    
    The check we actually want to do for disk over commit is: does the
    target host have enough free space to take the allocated size of this
    instance's disks? We leave checking over-allocation ratios to the
    scheduler. Note that if we use disk_available_least here and the
    destination host is over-allocated, this will always fail because free
    space will be negative, even though we're explicitly ok with that.
    
    Using disk_available_least would make sense for the non-overcommit
    case, where the test would be: would the target host have enough free
    space for this instance if the instance fully allocated all its
    storage, and everything else on the target also fully allocated its
    storage? As noted, we no longer actually run that test, though.
    
    We fix the issue for legacy microversions by fixing the destination
    host's reported disk space according to the given disk_over_commit
    parameter.
    
    Change-Id: I8a705114d47384fcd00955d4a4f204072fed57c2
    Resolves-bug: #1708572


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1708572

Title:
  Unable to live-migrate : Disk of instance is too large

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  os：centos7.3
  openstack：ocata

  when I　tried to live-migrate　an instance，it is wrong:
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 75, in wrapped
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     function_name, call_dict, binary)
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 66, in wrapped
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/utils.py", line 686, in decorated_function
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 216, in decorated_function
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 204, in decorated_function
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5281, in check_can_live_migrate_source
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     block_device_info)
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5750, in check_can_live_migrate_source
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     block_device_info)
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5874, in _assert_dest_node_has_enough_disk
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server     raise exception.MigrationPreCheckError(reason=reason)
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server MigrationPreCheckError: Migration pre-check error: Unable to migrate e9411b18-ce84-4094-8811-ad9675c245f3: Disk of instance is too large(available on destination host:-458487758848 < need:21692416)
  2017-07-28 11:33:03.917 18473 ERROR oslo_messaging.rpc.server.

  I found the free space of the destination host is -458487758848(A negative number).
  The same bug is in https://bugs.launchpad.net/mos/+bug/1650490
  But I don't kwon how to update my nova.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1708572/+subscriptions