yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1616938] [NEW] XenAPI: failed to create image from volume

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Jianghua Wang <wjh_fresh@xxxxxxx>
Date: Thu, 25 Aug 2016 14:02:03 -0000
Reply-to: Bug 1616938 <1616938@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

XenServer with dirver XenAPI, it always fails to create image from
volume.

Tempest test:
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_create_ebs_image_and_check_boot

2016-08-24 08:13:24.636 27347 DEBUG nova.compute.api [req-fd7afac4-a2ee-41cb-b785-02766806db26 tempest-TestVolumeBootPatternV2-1645487335 tempest-TestVolumeBootPatternV2-1645487335] [instance: 5d5a8b10-655c-457e-8ad9-edbfb6ecd278] Creating snapshot from volume 9953359c-327a-41bf-abec-f3c3da416390. snapshot_volume_backed /opt/stack/new/nova/nova/compute/api.py:2445^M
2016-08-24 08:13:25.964 27347 INFO os_vif [req-fd7afac4-a2ee-41cb-b785-02766806db26 tempest-TestVolumeBootPatternV2-1645487335 tempest-TestVolumeBootPatternV2-1645487335] Loaded VIF plugin class '<class 'vif_plug_ovs.ovs.OvsPlugin'>' with name 'ovs'^M
2016-08-24 08:13:25.965 27347 INFO os_vif [req-fd7afac4-a2ee-41cb-b785-02766806db26 tempest-TestVolumeBootPatternV2-1645487335 tempest-TestVolumeBootPatternV2-1645487335] Loaded VIF plugin class '<class 'vif_plug_linux_bridge.linux_bridge.LinuxBridgePlugin'>' with name 'linux_bridge'^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions [req-fd7afac4-a2ee-41cb-b785-02766806db26 tempest-TestVolumeBootPatternV2-1645487335 tempest-TestVolumeBootPatternV2-1645487335] Unexpected exception in API method^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions Traceback (most recent call last):^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/api/openstack/extensions.py", line 338, in wrapped^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/api/openstack/common.py", line 372, in inner^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/api/validation/__init__.py", line 73, in wrapper^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/api/validation/__init__.py", line 73, in wrapper^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/api/openstack/compute/servers.py", line 1072, in _action_create_image^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     metadata)^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/compute/api.py", line 146, in inner^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return f(self, context, instance, *args, **kw)^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/compute/api.py", line 2463, in snapshot_volume_backed^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return self.image_api.create(context, image_meta)^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/image/api.py", line 106, in create^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return session.create(context, image_info, data=data)^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/image/glance.py", line 626, in create^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     data, force_activate)^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/image/glance.py", line 658, in _create_v2^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     context, 2, 'create', **sent_service_image_meta)^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/image/glance.py", line 174, in call^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     result = getattr(client.images, method)(*args, **kwargs)^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/usr/local/lib/python2.7/dist-packages/glanceclient/v2/images.py", line 233, in create^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     raise TypeError(encodeutils.exception_to_unicode(e))^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions TypeError: Unable to set 'disk_format' to 'qcow2'. Reason: 'qcow2' is not one of [None, u'ami', u'ari', u'aki', u'vhd', u'raw', u'iso']^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions ^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions Failed validating u'enum' in schema[u'properties'][u'disk_format']:^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     {u'description': u'Format of the disk',^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions      u'enum': [None, u'ami', u'ari', u'aki', u'vhd', u'raw', u'iso'],^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions      u'type': [u'null', u'string']}^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions ^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions On instance[u'disk_format']:^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     'qcow2'^M
2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions ^M


Per my analysis, it should be bug due to it hardcode the disk format as qcow2. 
See: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L2471
At here, it cleans the disk_format and container_format

Then:

https://github.com/openstack/nova/blob/master//nova/image/glance.py#L651
It will hardcode the disk_format as qcow2 which is not supported by XenAPI.

nova/nova/image/glance.py
    def _create_v2(self, context, sent_service_image_meta, data=None,
                   force_activate=False):
        # Glance v1 allows image activation without setting disk and
        # container formats, v2 doesn't. It leads to the dirtiest workaround
        # where we have to hardcode this parameters.
        if force_activate:
            data = ''
            if 'disk_format' not in sent_service_image_meta:
                sent_service_image_meta['disk_format'] = 'qcow2'
            if 'container_format' not in sent_service_image_meta:
                sent_service_image_meta['container_format'] = 'bare'

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1616938

Title:
  XenAPI: failed to create image from volume

Status in OpenStack Compute (nova):
  New

Bug description:
  XenServer with dirver XenAPI, it always fails to create image from
  volume.

  Tempest test:
  tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_create_ebs_image_and_check_boot

  2016-08-24 08:13:24.636 27347 DEBUG nova.compute.api [req-fd7afac4-a2ee-41cb-b785-02766806db26 tempest-TestVolumeBootPatternV2-1645487335 tempest-TestVolumeBootPatternV2-1645487335] [instance: 5d5a8b10-655c-457e-8ad9-edbfb6ecd278] Creating snapshot from volume 9953359c-327a-41bf-abec-f3c3da416390. snapshot_volume_backed /opt/stack/new/nova/nova/compute/api.py:2445^M
  2016-08-24 08:13:25.964 27347 INFO os_vif [req-fd7afac4-a2ee-41cb-b785-02766806db26 tempest-TestVolumeBootPatternV2-1645487335 tempest-TestVolumeBootPatternV2-1645487335] Loaded VIF plugin class '<class 'vif_plug_ovs.ovs.OvsPlugin'>' with name 'ovs'^M
  2016-08-24 08:13:25.965 27347 INFO os_vif [req-fd7afac4-a2ee-41cb-b785-02766806db26 tempest-TestVolumeBootPatternV2-1645487335 tempest-TestVolumeBootPatternV2-1645487335] Loaded VIF plugin class '<class 'vif_plug_linux_bridge.linux_bridge.LinuxBridgePlugin'>' with name 'linux_bridge'^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions [req-fd7afac4-a2ee-41cb-b785-02766806db26 tempest-TestVolumeBootPatternV2-1645487335 tempest-TestVolumeBootPatternV2-1645487335] Unexpected exception in API method^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions Traceback (most recent call last):^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/api/openstack/extensions.py", line 338, in wrapped^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/api/openstack/common.py", line 372, in inner^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/api/validation/__init__.py", line 73, in wrapper^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/api/validation/__init__.py", line 73, in wrapper^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/api/openstack/compute/servers.py", line 1072, in _action_create_image^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     metadata)^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/compute/api.py", line 146, in inner^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return f(self, context, instance, *args, **kw)^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/compute/api.py", line 2463, in snapshot_volume_backed^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return self.image_api.create(context, image_meta)^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/image/api.py", line 106, in create^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     return session.create(context, image_info, data=data)^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/image/glance.py", line 626, in create^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     data, force_activate)^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/image/glance.py", line 658, in _create_v2^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     context, 2, 'create', **sent_service_image_meta)^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/image/glance.py", line 174, in call^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     result = getattr(client.images, method)(*args, **kwargs)^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions   File "/usr/local/lib/python2.7/dist-packages/glanceclient/v2/images.py", line 233, in create^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     raise TypeError(encodeutils.exception_to_unicode(e))^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions TypeError: Unable to set 'disk_format' to 'qcow2'. Reason: 'qcow2' is not one of [None, u'ami', u'ari', u'aki', u'vhd', u'raw', u'iso']^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions ^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions Failed validating u'enum' in schema[u'properties'][u'disk_format']:^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     {u'description': u'Format of the disk',^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions      u'enum': [None, u'ami', u'ari', u'aki', u'vhd', u'raw', u'iso'],^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions      u'type': [u'null', u'string']}^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions ^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions On instance[u'disk_format']:^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions     'qcow2'^M
  2016-08-24 08:13:26.049 27347 ERROR nova.api.openstack.extensions ^M


  Per my analysis, it should be bug due to it hardcode the disk format as qcow2. 
  See: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L2471
  At here, it cleans the disk_format and container_format

  Then:

  https://github.com/openstack/nova/blob/master//nova/image/glance.py#L651
  It will hardcode the disk_format as qcow2 which is not supported by XenAPI.

  nova/nova/image/glance.py
      def _create_v2(self, context, sent_service_image_meta, data=None,
                     force_activate=False):
          # Glance v1 allows image activation without setting disk and
          # container formats, v2 doesn't. It leads to the dirtiest workaround
          # where we have to hardcode this parameters.
          if force_activate:
              data = ''
              if 'disk_format' not in sent_service_image_meta:
                  sent_service_image_meta['disk_format'] = 'qcow2'
              if 'container_format' not in sent_service_image_meta:
                  sent_service_image_meta['container_format'] = 'bare'

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1616938/+subscriptions
Follow ups

[Bug 1616938] Re: XenAPI: failed to create image from volume backed instance with glance v2
From: Jianghua Wang, 2017-03-30
[Bug 1616938] Re: XenAPI: failed to create image from volume backed instance with glance v2
From: Jianghua Wang, 2017-03-30
[Bug 1616938] Re: XenAPI: failed to create image from volume backed instance with glance v2
From: OpenStack Infra, 2016-09-26
[Bug 1616938] Re: XenAPI: failed to create image from volume backed instance with glance v2
From: Matt Riedemann, 2016-09-24
[Bug 1616938] Re: XenAPI: failed to create image from volume backed instance with glance v2
From: Nikhil Komawar, 2016-09-24
[Bug 1616938] Re: XenAPI: failed to create image from volume backed instance
From: Matt Riedemann, 2016-09-24