yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1401021] [NEW] Glance quota fails when using copy-from

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Yukihiro KAWADA <1401021@xxxxxxxxxxxxxxxxxx>
Date: Wed, 10 Dec 2014 08:04:22 -0000
Reply-to: Bug 1401021 <1401021@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Glance quota fails when using copy-from.
Example, copying from swift.

to reproduce:
user_storage_quota=100GB  (value is any)

# glance --debug image-create --name demo_copy_from001 --disk-format
qcow2 --container-format bare --copy-from
https://objectstore.host.jp/v1/xxxzzzz/kwd001/test.qcow2 --is-public
False

to triage:
/usr/lib/python2.7/site-packages/glance_store
# git diff
diff --git a/_drivers/http.py b/_drivers/http.py
index e391c51..c9195d9 100644
--- a/_drivers/http.py
+++ b/_drivers/http.py
@@ -131,6 +131,12 @@ class Store(glance_store.driver.Store):
                 except StopIteration:
                     return ''

+            def read(self, i):
+                try:
+                    return self.wrapped.next()
+                except StopIteration:
+                    return ''

Error log:
2014-12-10 16:02:38.241 5296 DEBUG glance.api.v1.images [4d76d2f1-214e-4628-9503-f0ff7852940d 3be8a2df9ebe4bb4960cb7fe8c01fb16 8a5864e30e2b4019ad4c3ba359aa40dc - - -] Uploading image data for image 55f6b869-a56c-4fa4-80a2-2bcb80f7e9cb to file store _upload /usr/lib/python2.
7/site-packages/glance/api/v1/images.py:630
2014-12-10 16:02:38.377 5296 ERROR glance.api.v1.upload_utils [4d76d2f1-214e-4628-9503-f0ff7852940d 3be8a2df9ebe4bb4960cb7fe8c01fb16 8a5864e30e2b4019ad4c3ba359aa40dc - - -] Failed to upload image 55f6b869-a56c-4fa4-80a2-2bcb80f7e9cb
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils Traceback (most recent call last):
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance/api/v1/upload_utils.py", line 108, in upload_data_to_store
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     store)
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 342, in store_add_to_backend
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     (location, size, checksum, metadata) = store.add(image_id, data, size)
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance_store/_drivers/filesystem.py", line 461, in add
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     self._delete_partial(filepath, image_id)
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance_store/openstack/common/excutils.py", line 82, in __exit__
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     six.reraise(self.type_, self.value, self.tb)
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance_store/_drivers/filesystem.py", line 448, in add
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     self.WRITE_CHUNKSIZE):
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance_store/common/utils.py", line 67, in chunkiter
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     chunk = fp.read(chunk_size)
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 138, in readfn
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     result = fd.read(*args)
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 208, in read
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     result = self.data.read(i)
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils AttributeError: 'ResponseIndexable' object has no attribute 'read'
2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils

** Affects: glance
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1401021

Title:
  Glance quota fails when using copy-from

Status in OpenStack Image Registry and Delivery Service (Glance):
  New

Bug description:
  Glance quota fails when using copy-from.
  Example, copying from swift.

  to reproduce:
  user_storage_quota=100GB  (value is any)

  # glance --debug image-create --name demo_copy_from001 --disk-format
  qcow2 --container-format bare --copy-from
  https://objectstore.host.jp/v1/xxxzzzz/kwd001/test.qcow2 --is-public
  False

  to triage:
  /usr/lib/python2.7/site-packages/glance_store
  # git diff
  diff --git a/_drivers/http.py b/_drivers/http.py
  index e391c51..c9195d9 100644
  --- a/_drivers/http.py
  +++ b/_drivers/http.py
  @@ -131,6 +131,12 @@ class Store(glance_store.driver.Store):
                   except StopIteration:
                       return ''

  +            def read(self, i):
  +                try:
  +                    return self.wrapped.next()
  +                except StopIteration:
  +                    return ''

  Error log:
  2014-12-10 16:02:38.241 5296 DEBUG glance.api.v1.images [4d76d2f1-214e-4628-9503-f0ff7852940d 3be8a2df9ebe4bb4960cb7fe8c01fb16 8a5864e30e2b4019ad4c3ba359aa40dc - - -] Uploading image data for image 55f6b869-a56c-4fa4-80a2-2bcb80f7e9cb to file store _upload /usr/lib/python2.
  7/site-packages/glance/api/v1/images.py:630
  2014-12-10 16:02:38.377 5296 ERROR glance.api.v1.upload_utils [4d76d2f1-214e-4628-9503-f0ff7852940d 3be8a2df9ebe4bb4960cb7fe8c01fb16 8a5864e30e2b4019ad4c3ba359aa40dc - - -] Failed to upload image 55f6b869-a56c-4fa4-80a2-2bcb80f7e9cb
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils Traceback (most recent call last):
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance/api/v1/upload_utils.py", line 108, in upload_data_to_store
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     store)
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 342, in store_add_to_backend
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     (location, size, checksum, metadata) = store.add(image_id, data, size)
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance_store/_drivers/filesystem.py", line 461, in add
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     self._delete_partial(filepath, image_id)
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance_store/openstack/common/excutils.py", line 82, in __exit__
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     six.reraise(self.type_, self.value, self.tb)
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance_store/_drivers/filesystem.py", line 448, in add
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     self.WRITE_CHUNKSIZE):
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance_store/common/utils.py", line 67, in chunkiter
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     chunk = fp.read(chunk_size)
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 138, in readfn
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     result = fd.read(*args)
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 208, in read
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils     result = self.data.read(i)
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils AttributeError: 'ResponseIndexable' object has no attribute 'read'
  2014-12-10 16:02:38.377 5296 TRACE glance.api.v1.upload_utils

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1401021/+subscriptions
Follow ups

[Bug 1401021] [NEW] Glance quota fails when using copy-from
From: Yukihiro KAWADA, 2014-12-10
References

[Bug 1401021] [NEW] Glance quota fails when using copy-from
From: Yukihiro KAWADA, 2014-12-10