← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #57266

[Bug 1629371] [NEW] libvirt: volume snapshot stacktraces if the qemu guest agent is not available

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

The libvirt driver attempts to quiece the filesystem when doing a volume
snapshot and will stacktrace if the qemu guest agent is not available:

http://logs.openstack.org/86/147186/41/experimental/gate-tempest-dsvm-
full-devstack-plugin-nfs-
nv/fe58d89/logs/screen-n-cpu.txt.gz?level=TRACE#_2016-09-30_10_36_32_039

2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [req-93930c3c-4482-4c30-97ad-4dbcd251a9ba nova service] [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08] Unable to create quiesced VM snapshot, attempting again with quiescing disabled.
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08] Traceback (most recent call last):
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1827, in _volume_snapshot_create
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     reuse_ext=True, quiesce=True)
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 508, in snapshot
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     self._domain.snapshotCreateXML(conf.to_xml(), flags=flags)
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     rv = execute(f, *args, **kwargs)
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     six.reraise(c, e, tb)
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     rv = meth(*args, **kwargs)
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 2592, in snapshotCreateXML
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     if ret is None:raise libvirtError('virDomainSnapshotCreateXML() failed', dom=self)
2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08] libvirtError: 

The volume snapshot method then goes on to snapshot the guest without
quiescing first.

We shouldn't stacktrace an error on this, we should dump a warning in
the logs at most.

** Affects: nova
     Importance: Medium
     Assignee: Matt Riedemann (mriedem)
         Status: Triaged


** Tags: libvirt snapshot

** Changed in: nova
     Assignee: (unassigned) => Matt Riedemann (mriedem)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1629371

Title:
  libvirt: volume snapshot stacktraces if the qemu guest agent is not
  available

Status in OpenStack Compute (nova):
  Triaged

Bug description:
  The libvirt driver attempts to quiece the filesystem when doing a
  volume snapshot and will stacktrace if the qemu guest agent is not
  available:

  http://logs.openstack.org/86/147186/41/experimental/gate-tempest-dsvm-
  full-devstack-plugin-nfs-
  nv/fe58d89/logs/screen-n-cpu.txt.gz?level=TRACE#_2016-09-30_10_36_32_039

  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [req-93930c3c-4482-4c30-97ad-4dbcd251a9ba nova service] [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08] Unable to create quiesced VM snapshot, attempting again with quiescing disabled.
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08] Traceback (most recent call last):
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1827, in _volume_snapshot_create
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     reuse_ext=True, quiesce=True)
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 508, in snapshot
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     self._domain.snapshotCreateXML(conf.to_xml(), flags=flags)
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     result = proxy_call(self._autowrap, f, *args, **kwargs)
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     rv = execute(f, *args, **kwargs)
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     six.reraise(c, e, tb)
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     rv = meth(*args, **kwargs)
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]   File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 2592, in snapshotCreateXML
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08]     if ret is None:raise libvirtError('virDomainSnapshotCreateXML() failed', dom=self)
  2016-09-30 10:36:32.039 10790 ERROR nova.virt.libvirt.driver [instance: 01211c7e-b3ac-4d17-80bb-30fbe8494c08] libvirtError: 

  The volume snapshot method then goes on to snapshot the guest without
  quiescing first.

  We shouldn't stacktrace an error on this, we should dump a warning in
  the logs at most.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1629371/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next