← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1588657] Re: Nova shelve creates duplicated images in cells environment

 

** Changed in: nova/mitaka
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1588657

Title:
  Nova shelve creates duplicated images in cells environment

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) mitaka series:
  Fix Released

Bug description:
  Description
  ===========

  Nova shelve creates duplicated images in cells environment.
  I think, this behavior is caused by below patch.
  (not reproduced in icehouse version)

  commit 394ac5eea215831e82f96d43177442af67b40bb4

  I modified source code like below, and restart nova-api. It seems
  working fine.

  diff --git a/nova/compute/cells_api.py b/nova/compute/cells_api.py
  index 175ece2..4fe44a3 100644
  --- a/nova/compute/cells_api.py
  +++ b/nova/compute/cells_api.py
  @@ -322,8 +322,6 @@ class ComputeCellsAPI(compute_api.API):
       @check_instance_cell
       def shelve(self, context, instance, clean_shutdown=True):
           """Shelve the given instance."""
  -        super(ComputeCellsAPI, self).shelve(context, instance,
  -                clean_shutdown=clean_shutdown)
           self._cast_to_cells(context, instance, 'shelve',
                   clean_shutdown=clean_shutdown)

  Steps to reproduce
  ==================

  1. Create cells environment (ex. using devstack)

  2. Create vm by Nova

  openstack@localhost:~$ nova list
  +--------------------------------------+---------+--------+------------+-------------+--------------------------------+
  | ID                                   | Name    | Status | Task State | Power State | Networks                       |
  +--------------------------------------+---------+--------+------------+-------------+--------------------------------+
  | a520a4a7-6991-40e1-885a-bd3138c2b5dc | test-vm | ACTIVE | -          | Running     | public=2001:db8::4, 172.24.4.4 |
  +--------------------------------------+---------+--------+------------+-------------+--------------------------------+

  3. Execute nova shelve to vm

  openstack@localhost:~$ nova shelve test-vm
  openstack@localhost:~$ nova list
  +--------------------------------------+---------+-------------------+------------+-------------+--------------------------------+
  | ID                                   | Name    | Status            | Task State | Power State | Networks                       |
  +--------------------------------------+---------+-------------------+------------+-------------+--------------------------------+
  | a520a4a7-6991-40e1-885a-bd3138c2b5dc | test-vm | SHELVED_OFFLOADED | -          | Shutdown    | public=2001:db8::4, 172.24.4.4 |
  +--------------------------------------+---------+-------------------+------------+-------------+--------------------------------+

  4. Two images which is named <vmname>-shelved are created in glance

  openstack@localhost:~$ glance image-list
  +--------------------------------------+---------------------------------+
  | ID                                   | Name                            |
  +--------------------------------------+---------------------------------+
  | 3a106d4e-0b55-4e84-990a-a8aa4ae68c2a | cirros-0.3.4-x86_64-uec         |
  | ffc958ac-7435-42a4-9dc6-b42a7c518c2e | cirros-0.3.4-x86_64-uec-kernel  |
  | 51378b3f-86f6-4d70-a22e-768599eafc09 | cirros-0.3.4-x86_64-uec-ramdisk |
  | 42a6f2b0-c1b4-43d2-95c8-45e7c2d6ca3e | test-vm-shelved                 |
  | cc06acba-0911-49ec-86c2-d08c1084fbbe | test-vm-shelved                 |
  +--------------------------------------+---------------------------------+

  5. Only an glance image is used for nova-shelve.

  openstack@localhost:~$ glance image-show 42a6f2b0-c1b4-43d2-95c8-45e7c2d6ca3e
  +------------------+--------------------------------------+
  | Property         | Value                                |
  +------------------+--------------------------------------+
  | base_image_ref   | 3a106d4e-0b55-4e84-990a-a8aa4ae68c2a |
  | checksum         | None                                 |
  | container_format | ami                                  |
  | created_at       | 2016-06-03T06:16:32Z                 |
  | disk_format      | ami                                  |
  | id               | 42a6f2b0-c1b4-43d2-95c8-45e7c2d6ca3e |
  | image_type       | snapshot                             |
  | instance_uuid    | a520a4a7-6991-40e1-885a-bd3138c2b5dc |
  | kernel_id        | ffc958ac-7435-42a4-9dc6-b42a7c518c2e |
  | min_disk         | 1                                    |
  | min_ram          | 0                                    |
  | name             | test-vm-shelved                      |
  | owner            | 73af4afafeae4d5189d8190a8e876418     |
  | protected        | False                                |
  | ramdisk_id       | 51378b3f-86f6-4d70-a22e-768599eafc09 |
  | size             | 0                                    |
  | status           | queued                               |
  | tags             | []                                   |
  | updated_at       | 2016-06-03T06:16:32Z                 |
  | user_id          | c4954683e8244a0f84f509b8f3d63a4d     |
  | virtual_size     | None                                 |
  | visibility       | private                              |
  +------------------+--------------------------------------+
  openstack@localhost:~$ glance image-show cc06acba-0911-49ec-86c2-d08c1084fbbe
  +------------------+--------------------------------------+
  | Property         | Value                                |
  +------------------+--------------------------------------+
  | base_image_ref   | 3a106d4e-0b55-4e84-990a-a8aa4ae68c2a |
  | checksum         | ac94fabcc66952b36066eebc6af577d6     |
  | container_format | ami                                  |
  | created_at       | 2016-06-03T06:16:33Z                 |
  | disk_format      | ami                                  |
  | id               | cc06acba-0911-49ec-86c2-d08c1084fbbe |
  | image_location   | snapshot                             |
  | image_state      | available                            |
  | image_type       | snapshot                             |
  | instance_uuid    | a520a4a7-6991-40e1-885a-bd3138c2b5dc |
  | kernel_id        | ffc958ac-7435-42a4-9dc6-b42a7c518c2e |
  | min_disk         | 1                                    |
  | min_ram          | 0                                    |
  | name             | test-vm-shelved                      |
  | owner            | 73af4afafeae4d5189d8190a8e876418     |
  | owner_id         | 73af4afafeae4d5189d8190a8e876418     |
  | protected        | False                                |
  | ramdisk_id       | 51378b3f-86f6-4d70-a22e-768599eafc09 |
  | size             | 10682368                             |
  | status           | active                               |
  | tags             | []                                   |
  | updated_at       | 2016-06-03T06:17:02Z                 |
  | user_id          | c4954683e8244a0f84f509b8f3d63a4d     |
  | virtual_size     | None                                 |
  | visibility       | private                              |
  +------------------+--------------------------------------+

  Expected result
  ===============

  * Only an image is created

  openstack@localhost:~$ glance image-list
  +--------------------------------------+---------------------------------+
  | ID                                   | Name                            |
  +--------------------------------------+---------------------------------+
  | 3a106d4e-0b55-4e84-990a-a8aa4ae68c2a | cirros-0.3.4-x86_64-uec         |
  | ffc958ac-7435-42a4-9dc6-b42a7c518c2e | cirros-0.3.4-x86_64-uec-kernel  |
  | 51378b3f-86f6-4d70-a22e-768599eafc09 | cirros-0.3.4-x86_64-uec-ramdisk |
  | cc06acba-0911-49ec-86c2-d08c1084fbbe | test-vm-shelved                 |
  +--------------------------------------+---------------------------------+

  Actual result
  =============

  * Duplicated image is created on glance

  openstack@localhost:~$ glance image-list
  +--------------------------------------+---------------------------------+
  | ID                                   | Name                            |
  +--------------------------------------+---------------------------------+
  | 3a106d4e-0b55-4e84-990a-a8aa4ae68c2a | cirros-0.3.4-x86_64-uec         |
  | ffc958ac-7435-42a4-9dc6-b42a7c518c2e | cirros-0.3.4-x86_64-uec-kernel  |
  | 51378b3f-86f6-4d70-a22e-768599eafc09 | cirros-0.3.4-x86_64-uec-ramdisk |
  | 42a6f2b0-c1b4-43d2-95c8-45e7c2d6ca3e | test-vm-shelved                 |
  | cc06acba-0911-49ec-86c2-d08c1084fbbe | test-vm-shelved                 |
  +--------------------------------------+---------------------------------+

  Environment
  ===========

  * Devstack with cell
  ** This is snippet of localrc

  ENABLED_SERVICES=g-api,g-reg,key,n-api,n-cpu,n-cell,n-net,n-cond,n-sch,n-novnc,n-cauth,mysql,neutron,q-svc,q-agt,q-dhcp,q-l3,q-meta,horizon,s-proxy,s-object,s-container,s-account,rabbit

  * nova version
  ** This is on stable/liberty

  git log -1
  c5d0d2d0f4940d2e9f5888d7229fa4ac202ff161

  Logs & Configs
  ==============

  * Duplicated image is created on nova-api

  2016-06-03 02:16:32.537 DEBUG glanceclient.common.http [req-c7963aa0-f38a-4580-98b2-fc1464adabed admin admin] curl -g -i -X POST -H 'x-image-meta-property-user_id: c4954683e8244a0f84f509b8f3d63a4d' -H 'X-Auth-Token: {SHA1}0d0ff6398ca7d37
  52cf85f625b8009e3373c20af' -H 'Accept: */*' -H 'User-Agent: python-glanceclient' -H 'x-image-meta-name: test-vm-shelved' -H 'x-image-meta-container_format: ami' -H 'X-Tenant-Id: 73af4afafeae4d5189d8190a8e876418' -H 'x-image-meta-property
  -image_type: snapshot' -H 'X-User-Id: c4954683e8244a0f84f509b8f3d63a4d' -H 'x-image-meta-min_ram: 0' -H 'Accept-Encoding: gzip, deflate' -H 'x-image-meta-property-instance_uuid: a520a4a7-6991-40e1-885a-bd3138c2b5dc' -H 'x-image-meta-prop
  erty-ramdisk_id: 51378b3f-86f6-4d70-a22e-768599eafc09' -H 'Connection: keep-alive' -H 'x-image-meta-property-kernel_id: ffc958ac-7435-42a4-9dc6-b42a7c518c2e' -H 'x-image-meta-min_disk: 1' -H 'X-Roles: admin' -H 'x-image-meta-property-bas
  e_image_ref: 3a106d4e-0b55-4e84-990a-a8aa4ae68c2a' -H 'x-image-meta-is_public: False' -H 'X-Identity-Status: Confirmed' -H 'Content-Type: application/octet-stream' -H 'x-image-meta-disk_format: ami' http://192.168.56.101:9292/v1/images from (pid=16454) log_curl_request /usr/lib/python2.7/site-packages/glanceclient/common/http.py:189
  2016-06-03 02:16:32.770 DEBUG glanceclient.common.http [req-c7963aa0-f38a-4580-98b2-fc1464adabed admin admin]
  HTTP/1.1 201 Created
  Content-Length: 755
  Connection: keep-alive
  Location: http://192.168.56.101:9292/v1/images/42a6f2b0-c1b4-43d2-95c8-45e7c2d6ca3e
  Date: Fri, 03 Jun 2016 06:16:32 GMT
  Content-Type: application/json
  X-Openstack-Request-Id: req-d1bd0257-5771-4f4b-8ca2-25e1284664d9

  {"image": {"status": "queued", "deleted": false, "container_format": "ami", "min_ram": 0, "updated_at": "2016-06-03T06:16:32.000000", "owner": "73af4afafeae4d5189d8190a8e876418", "min_disk": 1, "is_public": false, "deleted_at": null, "id
  ": "42a6f2b0-c1b4-43d2-95c8-45e7c2d6ca3e", "size": 0, "virtual_size": null, "name": "test-vm-shelved", "checksum": null, "created_at": "2016-06-03T06:16:32.000000", "disk_format": "ami", "properties": {"instance_uuid": "a520a4a7-6991-40e
  1-885a-bd3138c2b5dc", "user_id": "c4954683e8244a0f84f509b8f3d63a4d", "image_type": "snapshot", "ramdisk_id": "51378b3f-86f6-4d70-a22e-768599eafc09", "kernel_id": "ffc958ac-7435-42a4-9dc6-b42a7c518c2e", "base_image_ref": "3a106d4e-0b55-4e
  84-990a-a8aa4ae68c2a"}, "protected": false}}

  * nova-cell(child) also creates the image

  2016-06-03 02:16:32.973 DEBUG glanceclient.common.http [req-c7963aa0-f38a-4580-98b2-fc1464adabed admin admin] curl -g -i -X POST -H 'x-image-meta-property-user_id: c4954683e8244a0f84f509b8f3d63a4d' -H 'X-Auth-Token: {SHA1}0d0ff6398ca7d37
  52cf85f625b8009e3373c20af' -H 'Accept: */*' -H 'User-Agent: python-glanceclient' -H 'x-image-meta-name: test-vm-shelved' -H 'x-image-meta-container_format: ami' -H 'X-Tenant-Id: 73af4afafeae4d5189d8190a8e876418' -H 'x-image-meta-property
  -image_type: snapshot' -H 'X-User-Id: c4954683e8244a0f84f509b8f3d63a4d' -H 'x-image-meta-min_ram: 0' -H 'Accept-Encoding: gzip, deflate' -H 'x-image-meta-property-instance_uuid: a520a4a7-6991-40e1-885a-bd3138c2b5dc' -H 'x-image-meta-prop
  erty-ramdisk_id: 51378b3f-86f6-4d70-a22e-768599eafc09' -H 'Connection: keep-alive' -H 'x-image-meta-property-kernel_id: ffc958ac-7435-42a4-9dc6-b42a7c518c2e' -H 'x-image-meta-min_disk: 1' -H 'X-Roles: admin' -H 'x-image-meta-property-bas
  e_image_ref: 3a106d4e-0b55-4e84-990a-a8aa4ae68c2a' -H 'x-image-meta-is_public: False' -H 'X-Identity-Status: Confirmed' -H 'Content-Type: application/octet-stream' -H 'x-image-meta-disk_format: ami' http://192.168.56.101:9292/v1/images f
  rom (pid=31995) log_curl_request /usr/lib/python2.7/site-packages/glanceclient/common/http.py:189
  2016-06-03 02:16:33.045 DEBUG glanceclient.common.http [req-c7963aa0-f38a-4580-98b2-fc1464adabed admin admin]
  HTTP/1.1 201 Created
  Content-Length: 755
  Connection: keep-alive
  Location: http://192.168.56.101:9292/v1/images/cc06acba-0911-49ec-86c2-d08c1084fbbe
  Date: Fri, 03 Jun 2016 06:16:33 GMT
  Content-Type: application/json
  X-Openstack-Request-Id: req-fa0bde4b-c741-4922-8c40-741d3a4576c2

  {"image": {"status": "queued", "deleted": false, "container_format": "ami", "min_ram": 0, "updated_at": "2016-06-03T06:16:33.000000", "owner": "73af4afafeae4d5189d8190a8e876418", "min_disk": 1, "is_public": false, "deleted_at": null, "id
  ": "cc06acba-0911-49ec-86c2-d08c1084fbbe", "size": 0, "virtual_size": null, "name": "test-vm-shelved", "checksum": null, "created_at": "2016-06-03T06:16:33.000000", "disk_format": "ami", "properties": {"instance_uuid": "a520a4a7-6991-40e
  1-885a-bd3138c2b5dc", "user_id": "c4954683e8244a0f84f509b8f3d63a4d", "image_type": "snapshot", "ramdisk_id": "51378b3f-86f6-4d70-a22e-768599eafc09", "kernel_id": "ffc958ac-7435-42a4-9dc6-b42a7c518c2e", "base_image_ref": "3a106d4e-0b55-4e
  84-990a-a8aa4ae68c2a"}, "protected": false}}

  * Config
  ** set 'glanceclient=DEBUG' to show glanceclient log on nova-cell

  nova-cells.conf

   [DEFAULT]
   default_log_levels=amqp=DEBUG,amqplib=DEBUG,boto=WARN,qpid=WARN,sqlalchemy=WARN,suds=INFO,oslo.messaging=DEBUG,iso8601=WARN,glanceclient=DEBUG

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1588657/+subscriptions


References