yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1596602] Re: Instance Snapshot failure: "Unable to set 'kernel_id' to 'None'. Reason: None is not of type u'string'"

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Matt Riedemann <mriedem@xxxxxxxxxx>
Date: Wed, 27 Jul 2016 17:01:56 -0000
Reply-to: Bug 1596602 <1596602@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
So the problem it looks like is we don't deploy this file:

https://raw.githubusercontent.com/openstack/glance/master/etc/schema-
image.json

To /etc/glance/ on our glance-api (controller) nodes. That defines the
schema for kernel_id and ramdisk_id (among other things). It's deployed
with devstack which is why we don't see issues in the upstream CI:

http://logs.openstack.org/77/335277/1/check/gate-tempest-dsvm-
full/de259ec/logs/etc/glance/schema-image.json.txt.gz

The schema we have in our cloud is this:

http://paste.openstack.org/show/542625/

With:

   "additionalProperties":{
      "type":"string"
   },

So if you pass kernel_id and ramdisk_id, they must not be None, but
that's what nova sends:

https://github.com/openstack/nova/blob/3f8076acdc7756b8a5f0f16d4885a47cb001483e/nova/image/glance.py#L844

Basically the glance API, or the image schema at least, is completely
configurable and nova isn't using the schema from glance to tell what
properties it can and can't set in the image body on the request to
glance.

So this is a bug in nova now to use the schema from glance.


** Changed in: python-glanceclient
       Status: Confirmed => Fix Released

** Also affects: nova
   Importance: Undecided
       Status: New

** Changed in: nova
       Status: New => Triaged

** Changed in: nova
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1596602

Title:
  Instance Snapshot failure: "Unable to set 'kernel_id' to 'None'.
  Reason: None is not of type u'string'"

Status in OpenStack Compute (nova):
  Triaged
Status in python-glanceclient:
  Fix Released

Bug description:
  Description
  ===========
  Attempted to take a snapshot of a suspended server through the CLI, but the command failed. (Nova log stack trace appended below)

  Steps to reproduce
  ==================
  1. Created a server instance:
     $ nova boot --image <image> --flavor m1.tiny snapshotvm
  2. Suspended the server:
     $ openstack server suspend <instance>
  3. Attempt to create server snapshot:
     $ nova image-create snapshotvm snapshotimage --poll

  Expected result
  ===============
  Expected to have a snapshot of my instance created in the image list.

  Actual result
  =============
  Received following error output from the image create command:

  # nova image-create snapshotvm snapshotimage --poll

  Server snapshotting... 25% complete
  ERROR (NotFound): Image not found. (HTTP 404) (Request-ID: req-4670eba3-a0d5-4814-b0a8-4aba37a1dd3a)

  Environment
  ===========
  1. Running from master level of Openstack

  2. Using KVM virtualization on Ubuntu 14.04:
  # kvm --version
  QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.24), Copyright (c) 2003-2008 Fabrice Bellard

  2. Which storage type did you use?
     LVM

  3. Which networking type did you use?
     Neutron


  2016-06-27 15:04:21.361 10781 INFO nova.compute.manager [req-f393be05-96c2-4d52-bfc1-0c5b644c553e d672d690e856437196a748ebc85abb41 b9b13412e4fd4fd6a6016368b720309e - - -] [instance: ab71489a-0716-404b-808a-165f2a85af74] Successfully reverted task state from image_uploading on failure for instance.
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server [req-f393be05-96c2-4d52-bfc1-0c5b644c553e d672d690e856437196a748ebc85abb41 b9b13412e4fd4fd6a6016368b720309e - - -] Exception during message handling
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/exception.py", line 104, in wrapped
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     payload)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 221, in __exit__
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 197, in force_reraise
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/exception.py", line 83, in wrapped
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 183, in decorated_function
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     LOG.warning(msg, e, instance=instance)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 221, in __exit__
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 197, in force_reraise
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 152, in decorated_function
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 211, in decorated_function
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 221, in __exit__
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 197, in force_reraise
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 199, in decorated_function
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 259, in decorated_function
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     instance=instance)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 221, in __exit__
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 197, in force_reraise
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 249, in decorated_function
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     *args, **kwargs)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 2983, in snapshot_instance
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     task_states.IMAGE_SNAPSHOT)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 3013, in _snapshot_instance
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     update_task_state)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1549, in snapshot
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     image_file)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/image/api.py", line 130, in update
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     purge_props=purge_props)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/image/glance.py", line 694, in update
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     _reraise_translated_image_exception(image_id)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/image/glance.py", line 1019, in _reraise_translated_image_exception
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     six.reraise(new_exc, None, exc_trace)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/image/glance.py", line 692, in update
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     image = self._update_v2(context, sent_service_image_meta, data)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/image/glance.py", line 702, in _update_v2
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     context, 2, 'update', **sent_service_image_meta)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/image/glance.py", line 174, in call
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     result = getattr(client.images, method)(*args, **kwargs)
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/glanceclient/v2/images.py", line 264, in update
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     raise TypeError(encodeutils.exception_to_unicode(e))
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server TypeError: Unable to set 'kernel_id' to 'None'. Reason: None is not of type u'string'
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server 
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server Failed validating u'type' in schema[u'additionalProperties']:
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     {u'type': u'string'}
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server 
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server On instance['kernel_id']:
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server     None
  2016-06-27 15:04:21.364 10781 ERROR oslo_messaging.rpc.server

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1596602/+subscriptions
References

[Bug 1596602] [NEW] Instance Snapshot failure
From: Ben VanHavermaet, 2016-06-27