yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1407592] Re: Snapshots fail to upload larger (~30G+) images, with error '500 Internal Server Error Failed to upload image'

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Morgan Fainberg <morgan.fainberg@xxxxxxxxx>
Date: Thu, 12 Mar 2015 21:59:34 -0000
Reply-to: Bug 1407592 <1407592@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
>From a Keystone perspective, this is strictly the token TTL (at best).
This can be resolved by increasing the TTL. We can evaluate changing the
defaults, but I am unsure how to calculate what the default *should* be
for all use cases. We will continue to iterate on this in general,
however, this is not a keystone bug per-say. So i am marking this as
wont fix within Keystone.

We should provide better documentation on the token TTL and re-evaluate
the defaults.

** Changed in: keystone
       Status: New => Won't Fix

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1407592

Title:
  Snapshots fail to upload larger (~30G+) images, with error '500
  Internal Server Error Failed to upload image'

Status in OpenStack Image Registry and Delivery Service (Glance):
  New
Status in OpenStack Identity (Keystone):
  Won't Fix
Status in OpenStack Compute (Nova):
  Incomplete

Bug description:
  I use Ubuntu 14.04.1 and OpenStack Juno latest release.

  In case I do SNAPSHOT for small virtual machine - it works.  In case I
  do SNAPSHOT for a big size VM (more than 30G) it fails after starting
  of saving SNAPSHOT to a Glance storage.

  So if to summarize:
  - could be created SNAPSHOTS for small VMs but not for a big
  - it is enough space on Glance storage and on root partitions on all involved physical machines

  The following errors could be found in logs:
  /var/log/nova/nova-compute.log
  2015-01-05 11:14:56.745 1767 INFO nova.compute.resource_tracker [-] Compute_service record updated for node4:node4
  2015-01-05 11:15:32.157 1767 ERROR glanceclient.common.http [req-692621ba-7a6b-471a-9e16-d74ee2eb875d ] Request returned failure status 500.
  2015-01-05 11:15:32.158 1767 WARNING urllib3.connectionpool [req-692621ba-7a6b-471a-9e16-d74ee2eb875d ] HttpConnectionPool is full, discarding connection: cloud
  2015-01-05 11:15:34.479 1767 ERROR glanceclient.common.http [req-692621ba-7a6b-471a-9e16-d74ee2eb875d ] Request returned failure status 401.
  2015-01-05 11:15:34.480 1767 ERROR nova.compute.manager [req-692621ba-7a6b-471a-9e16-d74ee2eb875d None] [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d] Error while trying to clean up image 71cd0c84-fa54-4d35-9ad6-63e36969de1c
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d] Traceback (most recent call last):
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 375, in decorated_function
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]     self.image_api.delete(context, image_id)
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]   File "/usr/lib/python2.7/dist-packages/nova/image/api.py", line 137, in delete
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]     return session.delete(context, image_id)
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]   File "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 418, in delete
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]     self._client.call(context, 1, 'delete', image_id)
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]   File "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 232, in call
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]     return getattr(client.images, method)(*args, **kwargs)
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]   File "/usr/lib/python2.7/dist-packages/glanceclient/v1/images.py", line 255, in delete
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]     resp, body = self.client.delete(url)
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]   File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 265, in delete
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]     return self._request('DELETE', url, **kwargs)
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]   File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 221, in _request
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]     raise exc.from_response(resp, resp.content)
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d] HTTPUnauthorized: <html>
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]  <head>
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]   <title>401 Unauthorized</title>
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]  </head>
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]  <body>
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]   <h1>401 Unauthorized</h1>
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]   This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required.<br /><br />
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]  </body>
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d] </html> (HTTP 401)
  2015-01-05 11:15:34.480 1767 TRACE nova.compute.manager [instance: f5a232a4-a7b2-4858-b793-38936b0f7a4d]
  2015-01-05 11:15:34.876 1767 INFO nova.scheduler.client.report [req-692621ba-7a6b-471a-9e16-d74ee2eb875d None] Compute_service record updated for ('node4', 'node4')
  2015-01-05 11:15:34.879 1767 ERROR oslo.messaging.rpc.dispatcher [req-692621ba-7a6b-471a-9e16-d74ee2eb875d ] Exception during message handling: <html>
   <head>
    <title>500 Internal Server Error</title>
   </head>
   <body>
    <h1>500 Internal Server Error</h1>
    Failed to upload image 71cd0c84-fa54-4d35-9ad6-63e36969de1c<br /><br />

   </body>
  </html> (HTTP 500)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     payload)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     return f(self, context, *args, **kw)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 303, in decorated_function
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     pass
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 289, in decorated_function
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 331, in decorated_function
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     kwargs['instance'], e, sys.exc_info())
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 319, in decorated_function
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 379, in decorated_function
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     instance=instance)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 369, in decorated_function
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     *args, **kwargs)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3027, in snapshot_instance
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     task_states.IMAGE_SNAPSHOT)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3058, in _snapshot_instance
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     update_task_state)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1752, in snapshot
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     image_file)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/image/api.py", line 126, in update
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     purge_props=purge_props)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 405, in update
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     _reraise_translated_image_exception(image_id)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 403, in update
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     image_id, **image_meta)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 232, in call
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     return getattr(client.images, method)(*args, **kwargs)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/glanceclient/v1/images.py", line 329, in update
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     resp, body = self.client.put(url, headers=hdrs, data=image_data)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 259, in put
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     return self._request('PUT', url, **kwargs)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 221, in _request
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher     raise exc.from_response(resp, resp.content)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher HTTPInternalServerError: <html>
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher  <head>
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   <title>500 Internal Server Error</title>
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher  </head>
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher  <body>
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   <h1>500 Internal Server Error</h1>
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher   Failed to upload image 71cd0c84-fa54-4d35-9ad6-63e36969de1c<br /><br />
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher  </body>
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher </html> (HTTP 500)
  2015-01-05 11:15:34.879 1767 TRACE oslo.messaging.rpc.dispatcher
  2015-01-05 11:15:55.562 1767 AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources

  /var/log/glance/api.log
  2015-01-05 11:15:20.569 19103 INFO glance.wsgi.server [-] 192.168.7.215 - - [05/Jan/2015 11:15:20] "HEAD /v1/images/2403f983-c6e2-4de1-9e56-dec0cbc1c203 HTTP/1.1" 200 837 0.014521
  2015-01-05 11:15:32.054 19103 INFO glance.registry.client.v1.client [-] Registry client request PUT /images/71cd0c84-fa54-4d35-9ad6-63e36969de1c raised NotAuthenticated
  2015-01-05 11:15:32.054 19103 ERROR glance.api.v1.upload_utils [-] Failed to upload image 71cd0c84-fa54-4d35-9ad6-63e36969de1c
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils Traceback (most recent call last):
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/api/v1/upload_utils.py", line 159, in upload_data_to_store
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils     from_state='saving')
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/registry/client/v1/api.py", line 169, in update_image_metadata
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils     from_state=from_state)
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/registry/client/v1/client.py", line 198, in update_image
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils     headers=headers)
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/registry/client/v1/client.py", line 130, in do_request
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils     'exc_name': exc_name})
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/openstack/common/excutils.py", line 82, in __exit__
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils     six.reraise(self.type_, self.value, self.tb)
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/registry/client/v1/client.py", line 115, in do_request
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils     **kwargs)
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 71, in wrapped
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils     return func(self, *args, **kwargs)
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 373, in do_request
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils     headers=copy.deepcopy(headers))
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 85, in wrapped
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils     return func(self, method, url, body, headers)
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 516, in _do_request
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils     raise exception.NotAuthenticated(res.read())
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils NotAuthenticated: Authentication required
  2015-01-05 11:15:32.054 19103 TRACE glance.api.v1.upload_utils
  2015-01-05 11:15:32.152 19103 INFO glance.registry.client.v1.client [-] Registry client request PUT /images/71cd0c84-fa54-4d35-9ad6-63e36969de1c raised NotAuthenticated
  2015-01-05 11:15:32.152 19103 ERROR glance.api.v1.upload_utils [-] Unable to kill image 71cd0c84-fa54-4d35-9ad6-63e36969de1c:
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils Traceback (most recent call last):
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/api/v1/upload_utils.py", line 77, in safe_kill
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils     _kill(req, image_id, from_state)
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/api/v1/upload_utils.py", line 62, in _kill
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils     from_state=from_state)
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/registry/client/v1/api.py", line 169, in update_image_metadata
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils     from_state=from_state)
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/registry/client/v1/client.py", line 198, in update_image
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils     headers=headers)
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/registry/client/v1/client.py", line 130, in do_request
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils     'exc_name': exc_name})
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/openstack/common/excutils.py", line 82, in __exit__
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils     six.reraise(self.type_, self.value, self.tb)
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/registry/client/v1/client.py", line 115, in do_request
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils     **kwargs)
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 71, in wrapped
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils     return func(self, *args, **kwargs)
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 373, in do_request
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils     headers=copy.deepcopy(headers))
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 85, in wrapped
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils     return func(self, method, url, body, headers)
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils   File "/usr/lib/python2.7/dist-packages/glance/common/client.py", line 516, in _do_request
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils     raise exception.NotAuthenticated(res.read())
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils NotAuthenticated: Authentication required
  2015-01-05 11:15:32.152 19103 TRACE glance.api.v1.upload_utils
  2015-01-05 11:15:32.154 19103 INFO glance.wsgi.server [-] 192.168.7.219 - - [05/Jan/2015 11:15:32] "PUT /v1/images/71cd0c84-fa54-4d35-9ad6-63e36969de1c HTTP/1.1" 500 407 700.615980
  2015-01-05 11:15:34.463 19103 INFO urllib3.connectionpool [-] Starting new HTTP connection (1): cloud
  2015-01-05 11:15:34.476 19103 WARNING keystonemiddleware.auth_token [-] Authorization failed for token
  2015-01-05 11:15:34.476 19103 WARNING keystonemiddleware.auth_token [-] Authorization failed for token
  2015-01-05 11:15:34.477 19103 INFO keystonemiddleware.auth_token [-] Invalid user token - deferring reject downstream
  2015-01-05 11:15:34.477 19103 INFO glance.wsgi.server [-] 192.168.7.219 - - [05/Jan/2015 11:15:34] "DELETE /v1/images/71cd0c84-fa54-4d35-9ad6-63e36969de1c HTTP/1.1" 401 485 0.015917

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1407592/+subscriptions
References

[Bug 1407592] [NEW] Snapshots fail to upload larger (~30G+) images, with error '500 Internal Server Error Failed to upload image'
From: Michael Zhdanov, 2015-01-05