← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #80026

[Bug 1844315] [NEW] Unable to upload images from external networks

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

Hi All,

We have a rocky OSA 18.1.2 branch setup. Image uploads to glance from remote networks are failing.
When we try try to upload from within the openstack networks, it is working fine, but from external networks, it is timing out and this is the error we observe in the glance-api logs:

----------------- 
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi [req-48abaf55-f91f-4962-b9bc-7dca62ae2e62 e62b971d129f67fee90791696d3b847ed790793378385e8b157f079c4aa025e2 4409fade1aee487bb92350d7cbdd91c3 - 0623aaad3fe64845b14d2ebaca620e16 0623aaad3fe64845b14d2ebaca620e16] Caught error: unable to receive chunked part: IOError: unable to receive chunked part
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi Traceback (most recent call last):
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/common/wsgi.py", line 1276, in __call__
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     request, **action_args)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/common/wsgi.py", line 1319, in dispatch
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     return method(*args, **kwargs)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/common/utils.py", line 417, in wrapped
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     return func(self, req, *args, **kwargs)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 279, in upload
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     self._restore(image_repo, image)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     self.force_reraise()
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     six.reraise(self.type_, self.value, self.tb)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 144, in upload
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     image.set_data(data, size, backend=backend)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/domain/proxy.py", line 198, in set_data
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     self.base.set_data(data, size, backend=backend)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/notifier.py", line 485, in set_data
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     _send_notification(notify_error, 'image.upload', msg)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     self.force_reraise()
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     six.reraise(self.type_, self.value, self.tb)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/notifier.py", line 432, in set_data
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     self.repo.set_data(data, size, backend=backend)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/api/policy.py", line 193, in set_data
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     return self.image.set_data(*args, **kwargs)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/quota/__init__.py", line 310, in set_data
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     self.image.set_data(data, size=size, backend=backend)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/location.py", line 483, in set_data
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     verifier=verifier)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/backend.py", line 541, in add_to_backend_with_multihash
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     image_id, data, size, hashing_algo, store, context, verifier)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/backend.py", line 518, in store_add_to_backend_with_multihash
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     image_id, data, size, hashing_algo, context=context, verifier=verifier)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/driver.py", line 274, in add_adapter
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     metadata_dict) = store_add_fun(*args, **kwargs)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/capabilities.py", line 225, in op_checker
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     return store_op_fun(store, *args, **kwargs)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/_drivers/filesystem.py", line 714, in add
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     raise errors.get(e.errno, e)
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi IOError: unable to receive chunked part
2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi
2019-09-17 11:50:52.488 6821 CRITICAL glance [req-48abaf55-f91f-4962-b9bc-7dca62ae2e62 e62b971d129f67fee90791696d3b847ed790793378385e8b157f079c4aa025e2 4409fade1aee487bb92350d7cbdd91c3 - 0623aaad3fe64845b14d2ebaca620e16 0623aaad3fe64845b14d2ebaca620e16] Unhandled error: IOError: write error
2019-09-17 11:50:52.488 6821 ERROR glance IOError: write error
2019-09-17 11:50:52.488 6821 ERROR glance
--------------------- 


All other api commands r working fine remotely, but only these uploads are failing. Ideally we suppose, openstack should allow remote uploads.


Thanks
Kumar

** Affects: glance
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1844315

Title:
  Unable to upload images from external networks

Status in Glance:
  New

Bug description:
  Hi All,

  We have a rocky OSA 18.1.2 branch setup. Image uploads to glance from remote networks are failing.
  When we try try to upload from within the openstack networks, it is working fine, but from external networks, it is timing out and this is the error we observe in the glance-api logs:

  ----------------- 
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi [req-48abaf55-f91f-4962-b9bc-7dca62ae2e62 e62b971d129f67fee90791696d3b847ed790793378385e8b157f079c4aa025e2 4409fade1aee487bb92350d7cbdd91c3 - 0623aaad3fe64845b14d2ebaca620e16 0623aaad3fe64845b14d2ebaca620e16] Caught error: unable to receive chunked part: IOError: unable to receive chunked part
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi Traceback (most recent call last):
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/common/wsgi.py", line 1276, in __call__
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     request, **action_args)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/common/wsgi.py", line 1319, in dispatch
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     return method(*args, **kwargs)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/common/utils.py", line 417, in wrapped
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     return func(self, req, *args, **kwargs)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 279, in upload
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     self._restore(image_repo, image)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     self.force_reraise()
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     six.reraise(self.type_, self.value, self.tb)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 144, in upload
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     image.set_data(data, size, backend=backend)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/domain/proxy.py", line 198, in set_data
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     self.base.set_data(data, size, backend=backend)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/notifier.py", line 485, in set_data
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     _send_notification(notify_error, 'image.upload', msg)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     self.force_reraise()
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     six.reraise(self.type_, self.value, self.tb)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/notifier.py", line 432, in set_data
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     self.repo.set_data(data, size, backend=backend)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/api/policy.py", line 193, in set_data
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     return self.image.set_data(*args, **kwargs)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/quota/__init__.py", line 310, in set_data
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     self.image.set_data(data, size=size, backend=backend)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance/location.py", line 483, in set_data
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     verifier=verifier)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/backend.py", line 541, in add_to_backend_with_multihash
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     image_id, data, size, hashing_algo, store, context, verifier)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/backend.py", line 518, in store_add_to_backend_with_multihash
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     image_id, data, size, hashing_algo, context=context, verifier=verifier)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/driver.py", line 274, in add_adapter
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     metadata_dict) = store_add_fun(*args, **kwargs)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/capabilities.py", line 225, in op_checker
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     return store_op_fun(store, *args, **kwargs)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi   File "/openstack/venvs/glance-18.1.2/lib/python2.7/site-packages/glance_store/_drivers/filesystem.py", line 714, in add
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi     raise errors.get(e.errno, e)
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi IOError: unable to receive chunked part
  2019-09-17 11:50:52.482 6821 ERROR glance.common.wsgi
  2019-09-17 11:50:52.488 6821 CRITICAL glance [req-48abaf55-f91f-4962-b9bc-7dca62ae2e62 e62b971d129f67fee90791696d3b847ed790793378385e8b157f079c4aa025e2 4409fade1aee487bb92350d7cbdd91c3 - 0623aaad3fe64845b14d2ebaca620e16 0623aaad3fe64845b14d2ebaca620e16] Unhandled error: IOError: write error
  2019-09-17 11:50:52.488 6821 ERROR glance IOError: write error
  2019-09-17 11:50:52.488 6821 ERROR glance
  --------------------- 

  
  All other api commands r working fine remotely, but only these uploads are failing. Ideally we suppose, openstack should allow remote uploads.

  
  Thanks
  Kumar

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1844315/+subscriptions
  Thread Previous • Date Previous • • Thread Next