yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1761062] Re: clean source instance directory failed in _cleanup_resize when images_type is rbd

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Launchpad Bug Tracker <1761062@xxxxxxxxxxxxxxxxxx>
Date: Wed, 12 Dec 2018 04:17:36 -0000
Reply-to: Bug 1761062 <1761062@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
[Expired for OpenStack Compute (nova) because there has been no activity
for 60 days.]

** Changed in: nova
       Status: Incomplete => Expired

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1761062

Title:
   clean source instance directory failed in _cleanup_resize when
  images_type is rbd

Status in OpenStack Compute (nova):
  Expired

Bug description:
  Description
  ===========
  When images_type is rbd, and boot an instance from image, perform resize and resize-confirm of this instance, it will not clean up source instance directory and result in live-migration this instance failed. The following is the error log. 

  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server [req-e20743e3-a683-41ba-b47b-ba92e97eff37 4f7cd8bf676d43bc9faf09b2eb41482f 2c3d8251c39545cbb6f77f331b7164f8 - default default] Exception during message handling: DestinationDiskExists: The supplied disk path (/var/lib/nova/instances/d8db3f2a-cd8f-48e1-9951-012d762664b2) already exists, it is expected not to exist.
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 76, in wrapped
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     function_name, call_dict, binary)
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 67, in wrapped
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/utils.py", line 880, in decorated_function
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 223, in decorated_function
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 211, in decorated_function
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5615, in pre_live_migration
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     migrate_data)
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova_patch/virt/libvirt/driver.py", line 1095, in wrap
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     migrate_data=migrate_data)
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7081, in pre_live_migration
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server     raise exception.DestinationDiskExists(path=instance_dir)
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server DestinationDiskExists: The supplied disk path (/var/lib/nova/instances/d8db3f2a-cd8f-48e1-9951-012d762664b2) already exists, it is expected not to exist.
  2018-03-22 10:05:40.657 20779 ERROR oslo_messaging.rpc.server
  2018-03-22 10:05:42.665 20779 INFO nova.virt.libvirt.driver [req-e20743e3-a683-41ba-b47b-ba92e97eff37 4f7cd8bf676d43bc9faf09b2eb41482f 2c3d8251c39545cbb6f77f331b7164f8 - default default] [instance: d8db3f2a-cd8f-48e1-9951-012d762664b2] Instance destroyed successfully.

  when images_type is rbd, it will considered is shared storage(in _is_storage_shared_with) in resize process.
  In this environment, instance directory is not shared storage actually.

  
  Environment
  ===========
  nova: origin/stable/pike
  libvirt+kvm

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761062/+subscriptions
References

[Bug 1761062] [NEW] clean source instance directory failed in _cleanup_resize when images_type is rbd
From: guolidong, 2018-04-04