yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #61283
[Bug 1662245] Re: Snapshoting fails in Mitaka
> I'm not sure if this bug should be recorded with Nova or Ubuntu..
Thanks for the report, IMHO this needs to be addressed by Ubuntu or who
ever is providing libvirt packages as these apparmor profiles are not
shipped within the nova project.
** Changed in: nova
Status: New => Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1662245
Title:
Snapshoting fails in Mitaka
Status in OpenStack Compute (nova):
Invalid
Bug description:
Description
===========
Running Mitaka installed on Ubuntu 14.04 (upgraded from Juno all the way to Mitaka). The dashboard reports success even though no snapshot is created.
Steps to reproduce
==================
Take snapshot using dashboard or the Openstack client.
Expected result
===============
New snapshot gets created, image registered in Glance.
Actual result
=============
Dashboard reports success right away, the image shows up in the images tab in Dashboard shortly before changing status to "Deleted" and disappearing.
The following message gets logged in "/var/log/nova/nova-compute.log".
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher [req-81b0372a-b2d4-4e30-b0bf-1f54de71f7d9 1344234bf2e54520bbcc9585e442ce78 2fb7b98d74134a37b9fba88856f60b78 - - -] Exception during message handling: internal error: unable to execute QEMU command 'migrate': Migration disabled: failed to allocate shared memory
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher incoming.message))
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 185, in _dispatch
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 110, in wrapped
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher payload)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher self.force_reraise()
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 89, in wrapped
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 359, in decorated_function
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher LOG.warning(msg, e, instance=instance)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher self.force_reraise()
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 328, in decorated_function
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 387, in decorated_function
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher self.force_reraise()
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 375, in decorated_function
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 435, in decorated_function
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher instance=instance)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher self.force_reraise()
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 425, in decorated_function
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher *args, **kwargs)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3158, in snapshot_instance
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher task_states.IMAGE_SNAPSHOT)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3188, in _snapshot_instance
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher update_task_state)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1667, in snapshot
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher instance)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1760, in _prepare_domain_for_snapshot
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher self.suspend(context, instance)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2649, in suspend
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher guest.save_memory_state()
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 379, in save_memory_state
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher self._domain.managedSave(0)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher rv = execute(f, *args, **kwargs)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher six.reraise(c, e, tb)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher rv = meth(*args, **kwargs)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/libvirt.py", line 1403, in managedSave
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher if ret == -1: raise libvirtError ('virDomainManagedSave() failed', dom=self)
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher libvirtError: internal error: unable to execute QEMU command 'migrate': Migration disabled: failed to allocate shared memory
2017-02-06 10:51:36.258 12196 ERROR oslo_messaging.rpc.dispatcher
Environment
===========
root@compute9-r2:~# dpkg -l | egrep "qemu|libvirt|kvm|nova"
ii ipxe-qemu 1.0.0+git-20131111.c3d1e78-2ubuntu1.1 all PXE boot firmware - ROM images for qemu
ii libvirt-bin 1.3.1-1ubuntu10.6~cloud0 amd64 programs for the libvirt library
ii libvirt0:amd64 1.3.1-1ubuntu10.6~cloud0 amd64 library for interfacing with different virtualization systems
ii nova-common 2:13.1.2-0ubuntu2~cloud0 all OpenStack Compute - common files
ii nova-compute 2:13.1.2-0ubuntu2~cloud0 all OpenStack Compute - compute node base
ii nova-compute-kvm 2:13.1.2-0ubuntu2~cloud0 all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 2:13.1.2-0ubuntu2~cloud0 all OpenStack Compute - compute node libvirt support
ii python-libvirt 1.3.1-1ubuntu1~cloud0 amd64 libvirt Python bindings
ii python-nova 2:13.1.2-0ubuntu2~cloud0 all OpenStack Compute Python libraries
ii python-novaclient 2:3.3.1-2ubuntu1~cloud0 all client library for OpenStack Compute API - Python 2.7
ii qemu-block-extra:amd64 1:2.5+dfsg-5ubuntu10.5~cloud0 amd64 extra block backend modules for qemu-system and qemu-utils
ii qemu-slof 20151103+dfsg-1ubuntu1~cloud0 all Slimline Open Firmware -- QEMU PowerPC version
ii qemu-system 1:2.5+dfsg-5ubuntu10.5~cloud0 amd64 QEMU full system emulation binaries
ii qemu-system-arm 1:2.5+dfsg-5ubuntu10.5~cloud0 amd64 QEMU full system emulation binaries (arm)
ii qemu-system-common 1:2.5+dfsg-5ubuntu10.5~cloud0 amd64 QEMU full system emulation binaries (common files)
ii qemu-system-mips 1:2.5+dfsg-5ubuntu10.5~cloud0 amd64 QEMU full system emulation binaries (mips)
ii qemu-system-misc 1:2.5+dfsg-5ubuntu10.5~cloud0 amd64 QEMU full system emulation binaries (miscelaneous)
ii qemu-system-ppc 1:2.5+dfsg-5ubuntu10.5~cloud0 amd64 QEMU full system emulation binaries (ppc)
ii qemu-system-sparc 1:2.5+dfsg-5ubuntu10.5~cloud0 amd64 QEMU full system emulation binaries (sparc)
ii qemu-system-x86 1:2.5+dfsg-5ubuntu10.5~cloud0 amd64 QEMU full system emulation binaries (x86)
ii qemu-utils 1:2.5+dfsg-5ubuntu10.5~cloud0 amd64 QEMU utilities
Researching the error message I found https://bugs.launchpad.net/fuel/+bug/1638269.
After I applied the patch https://review.fuel-
infra.org/#/c/28086/2/debian/apparmor/libvirt-qemu and restarting the
instance, the snapshot is working fine.
I'm not sure if this bug should be recorded with Nova or Ubuntu..
Thank you.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1662245/+subscriptions
References
