← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1606707] Re: instance snapshot fails with "AttributeError: size" when using glance v1

 

Reviewed:  https://review.openstack.org/347571
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=7a16c754fb49615b802e6b8f3d886847c168bd2a
Submitter: Jenkins
Branch:    master

commit 7a16c754fb49615b802e6b8f3d886847c168bd2a
Author: Matt Riedemann <mriedem@xxxxxxxxxx>
Date:   Tue Jul 26 18:27:48 2016 -0400

    Default image.size to 0 when extracting v1 image attributes
    
    When we snapshot a non-volume-backed instance, we create an
    image in nova.compute.api.API._create_image and set some
    values from the instance but 'size' isn't one of them.
    
    Later in the virt driver's snapshot method, at least for
    libvirt, it gets the snapshot image from the image API (glance)
    and when using glance v1 (use_glance_v1=True) the _extract_attributes
    method in nova.image.glance pulls the attributes out of the v1 image
    response to the form that nova expects. This code assumes that
    the 'size' attribute is set on the image, which for a snapshot image
    it might not be (yet anyway). This results in an AttributeError.
    
    This change defaults the size attribute value to 0 if it's not set.
    If it is set, but to None, we still use 0 as before.
    
    Change-Id: I14b0e44a7268231c2b19f013b563f0b8f09c2e88
    Closes-Bug: #1606707


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1606707

Title:
  instance snapshot fails with "AttributeError: size" when using glance
  v1

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  This is on a newton server deployed on 7/25 from master (newton).

  I set use_glance_v1=True in nova.conf on the compute node.

  I created a server from this image:

  +------------------+----------------------------------------------------------------------------------+
  | Property         | Value                                                                            |
  +------------------+----------------------------------------------------------------------------------+
  | checksum         | ee1eca47dc88f4879d8a229cc70a07c6                                                 |
  | container_format | bare                                                                             |
  | created_at       | 2016-07-06T17:25:08Z                                                             |
  | disk_format      | qcow2                                                                            |
  | id               | aac0314e-9bdf-4cee-a3d8-5f089008ea96                                             |
  | locations        | [{"url": "file:///var/lib/glance/images/aac0314e-9bdf-4cee-a3d8-5f089008ea96",   |
  |                  | "metadata": {}}]                                                                 |
  | min_disk         | 0                                                                                |
  | min_ram          | 0                                                                                |
  | name             | cirros                                                                           |
  | owner            | None                                                                             |
  | protected        | False                                                                            |
  | size             | 13287936                                                                         |
  | status           | active                                                                           |
  | tags             | []                                                                               |
  | updated_at       | 2016-07-06T17:25:09Z                                                             |
  | virtual_size     | None                                                                             |
  | visibility       | public                                                                           |
  +------------------+----------------------------------------------------------------------------------+

  
  I tried to snapshot the instance and it failed with this:

  http://paste.openstack.org/show/542180/

  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd] Traceback (most recent call last):
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 231, in decorated_function
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]     *args, **kwargs)
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 3024, in snapshot_instance
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]     task_states.IMAGE_SNAPSHOT)
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 3054, in _snapshot_instance
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]     update_task_state)
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1482, in snapshot
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]     snapshot = self._image_api.get(context, image_id)
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/image/api.py", line 93, in get
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]     show_deleted=show_deleted)
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/image/glance.py", line 266, in show
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]     include_locations=include_locations)
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/image/glance.py", line 878, in _translate_from_glance
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]     image, include_locations=include_locations)
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/nova/image/glance.py", line 968, in _extract_attributes
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]     output[attr] = getattr(image, attr) or 0
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/glanceclient/openstack/common/apiclient/base.py", line 491, in __getattr__
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]     return self.__getattr__(k)
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]   File "/opt/bbc/openstack-11.0-master/nova/local/lib/python2.7/site-packages/glanceclient/openstack/common/apiclient/base.py", line 493, in __getattr__
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]     raise AttributeError(k)
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd] AttributeError: size
  2016-07-26 21:25:34.563 51383 ERROR nova.compute.manager [instance: 85bcc918-4d00-4d21-82fe-65b13035adcd]

  Looking at that code, it looks like it assumes the 'size' attribute
  will be on the image, which it's not. Nova creates this image from the
  compute API:

  https://github.com/openstack/nova/blob/9326c1ed403477d627fa1b94e6937c99deed9ecd/nova/compute/api.py#L2165

  The extra_properties in this case come from the REST API, and I'm not
  passing any in in my snapshot request.

  The other properties come from the instance's system_metadata, of
  which image_size isn't set:

  mysql> select * from nova.instance_system_metadata where instance_uuid='85bcc918-4d00-4d21-82fe-65b13035adcd';
  +---------------------+------------+------------+----+--------------------------------------+------------------------+--------------------------------------+---------+
  | created_at          | updated_at | deleted_at | id | instance_uuid                        | key                    | value                                | deleted |
  +---------------------+------------+------------+----+--------------------------------------+------------------------+--------------------------------------+---------+
  | 2016-07-26 20:54:08 | NULL       | NULL       | 92 | 85bcc918-4d00-4d21-82fe-65b13035adcd | image_disk_format      | qcow2                                |       0 |
  | 2016-07-26 20:54:08 | NULL       | NULL       | 93 | 85bcc918-4d00-4d21-82fe-65b13035adcd | image_min_ram          | 0                                    |       0 |
  | 2016-07-26 20:54:08 | NULL       | NULL       | 94 | 85bcc918-4d00-4d21-82fe-65b13035adcd | image_min_disk         | 10                                   |       0 |
  | 2016-07-26 20:54:08 | NULL       | NULL       | 95 | 85bcc918-4d00-4d21-82fe-65b13035adcd | image_base_image_ref   | aac0314e-9bdf-4cee-a3d8-5f089008ea96 |       0 |
  | 2016-07-26 20:54:08 | NULL       | NULL       | 96 | 85bcc918-4d00-4d21-82fe-65b13035adcd | image_container_format | bare                                 |       0 |
  +---------------------+------------+------------+----+--------------------------------------+------------------------+--------------------------------------+---------+

  So it looks like we shouldn't rely on image.size being set when using
  glance v1 and taking a snapshot.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1606707/+subscriptions


References