yahoo-eng-team team mailing list archive

[Abhishek Kekane <abhishek.kekane@xxxxxxxxxxx>]

I have confirmed that this is not a failure any more by running these
tests in 6 different terminals and continuously 1 hour using "while
true; do tox -e py27 glance.tests.unit.v1.test_api || break; done"


** Changed in: glance
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1521756

Title:
  race condition in test_upload_image_http_nonexistent_location_url:
  glance_store doesn't handle properly invalid HTTP reply

Status in Glance:
  Invalid

Bug description:
  Victor asked for me to have a look at an intermittent failure he was
  seeing in https://review.openstack.org/#/c/250083/

  it shows up like so:
  Traceback (most recent call last):
    File "/home/robertc/work/openstack/glance/.tox/py34/lib/python3.4/site-packages/eventlet/greenpool.py", line 82, in _spawn_n_impl
      func(*args, **kwargs)
    File "/home/robertc/work/openstack/glance/glance/api/v1/images.py", line 737, in _upload_and_activate
      location_data = self._upload(req, image_meta)
    File "/home/robertc/work/openstack/glance/glance/api/v1/images.py", line 671, in _upload
      {'status': 'saving'})
    File "/home/robertc/work/openstack/glance/glance/registry/client/v1/api.py", line 174, in update_image_metadata
      from_state=from_state)
    File "/home/robertc/work/openstack/glance/glance/registry/client/v1/client.py", line 209, in update_image
      headers=headers)
    File "/home/robertc/work/openstack/glance/glance/registry/client/v1/client.py", line 141, in do_request
      'exc_name': exc_name})
    File "/home/robertc/work/openstack/glance/.tox/py34/lib/python3.4/site-packages/oslo_utils/excutils.py", line 204, in __exit__
      six.reraise(self.type_, self.value, self.tb)
    File "/home/robertc/work/openstack/glance/.tox/py34/lib/python3.4/site-packages/six.py", line 686, in reraise
      raise value
    File "/home/robertc/work/openstack/glance/glance/registry/client/v1/client.py", line 124, in do_request
      **kwargs)
    File "/home/robertc/work/openstack/glance/glance/common/client.py", line 71, in wrapped
      return func(self, *args, **kwargs)
    File "/home/robertc/work/openstack/glance/glance/common/client.py", line 375, in do_request
      headers=copy.deepcopy(headers))
    File "/home/robertc/work/openstack/glance/glance/common/client.py", line 88, in wrapped
      return func(self, method, url, body, headers)
    File "/home/robertc/work/openstack/glance/glance/common/client.py", line 524, in _do_request
      raise exception.NotFound(res.read())
  glance.common.exception.NotFound: b'Image not found'
  ======================================================================
  FAIL: glance.tests.unit.v1.test_api.TestGlanceAPI.test_upload_image_http_nonexistent_location_url
  tags: worker-0
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/home/robertc/work/openstack/glance/glance/tests/unit/v1/test_api.py", line 1149, in test_upload_image_http_nonexistent_location_url
      self.assertEqual(404, res.status_int)
    File "/home/robertc/work/openstack/glance/.tox/py34/lib/python3.4/site-packages/testtools/testcase.py", line 350, in assertEqual
      self.assertThat(observed, matcher, message)
    File "/home/robertc/work/openstack/glance/.tox/py34/lib/python3.4/site-packages/testtools/testcase.py", line 435, in assertThat
      raise mismatch_error
  testtools.matchers._impl.MismatchError: 404 != 201

  
  and through bisection I can reproduce it with 75 tests - I'm working on shrinking the set but it takes a couple hundred runs to be sure its a false branch, so its not super fast.

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1521756/+subscriptions