openstack team mailing list archive

[Antonio Messina <antonio.s.messina@xxxxxxxxx>]

Sorry, I think I have found the problem: one of the disk of the swift
server seems to have problems.

.a.

On Mon, Feb 11, 2013 at 5:11 PM, Antonio Messina
<antonio.s.messina@xxxxxxxxx> wrote:
> Hi all,
>
> We have a folsom installation on ubuntu, and we are having trouble
> creating snapshots. Sometimes (quite often indeed) the process fails
> and the snapshot image goes in "killed" status. This happen with both
> the Web frontend and nova image-create client. The funny thing is that
> creation of a snapshot on the very same instance can fail a few times,
> but it will eventually succeed.
>
> The nova-compute.log shows:
>
> 2013-02-11 17:03:54 DEBUG nova.utils
> [req-a789eaef-d7f7-4f41-807a-a7b910471a0b
> c5c6eb3e0ea8456c98acd07100a627af c4f9ddbce4f8444c896722e7c672d945]
> Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf
> qemu-img snapshot -c 656baf75ff104768bce7169233cb76d0
> /var/lib/nova/instances/instance-0000045c/disk execute
> /usr/lib/python2.7/dist-packages/nova/utils.py:183
> 2013-02-11 17:03:55 DEBUG nova.utils
> [req-a789eaef-d7f7-4f41-807a-a7b910471a0b
> c5c6eb3e0ea8456c98acd07100a627af c4f9ddbce4f8444c896722e7c672d945]
> Result was 0 execute
> /usr/lib/python2.7/dist-packages/nova/utils.py:199
> 2013-02-11 17:03:55 DEBUG nova.utils
> [req-a789eaef-d7f7-4f41-807a-a7b910471a0b
> c5c6eb3e0ea8456c98acd07100a627af c4f9ddbce4f8444c896722e7c672d945]
> Running cmd (subprocess): qemu-img convert -f qcow2 -O qcow2 -s
> 656baf75ff104768bce7169233cb76d0
> /var/lib/nova/instances/instance-0000045c/disk
> /var/lib/nova/instances/snapshots/tmpQYvmFD/656baf75ff104768bce7169233cb76d0
> execute /usr/lib/python2.7/dist-packages/nova/utils.py:183
> 2013-02-11 17:03:58 DEBUG nova.utils
> [req-a789eaef-d7f7-4f41-807a-a7b910471a0b
> c5c6eb3e0ea8456c98acd07100a627af c4f9ddbce4f8444c896722e7c672d945]
> Result was 0 execute
> /usr/lib/python2.7/dist-packages/nova/utils.py:199
> 2013-02-11 17:03:58 DEBUG nova.utils
> [req-a789eaef-d7f7-4f41-807a-a7b910471a0b
> c5c6eb3e0ea8456c98acd07100a627af c4f9ddbce4f8444c896722e7c672d945]
> Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf
> qemu-img snapshot -d 656baf75ff104768bce7169233cb76d0
> /var/lib/nova/instances/instance-0000045c/disk execute
> /usr/lib/python2.7/dist-packages/nova/utils.py:183
> 2013-02-11 17:03:58 DEBUG nova.utils
> [req-a789eaef-d7f7-4f41-807a-a7b910471a0b
> c5c6eb3e0ea8456c98acd07100a627af c4f9ddbce4f8444c896722e7c672d945]
> Result was 0 execute
> /usr/lib/python2.7/dist-packages/nova/utils.py:199
> 2013-02-11 17:04:01 DEBUG nova.utils
> [req-a789eaef-d7f7-4f41-807a-a7b910471a0b
> c5c6eb3e0ea8456c98acd07100a627af c4f9ddbce4f8444c896722e7c672d945]
> Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf
> tee /sys/class/net/vnet0/brport/hairpin_mode execute
> /usr/lib/python2.7/dist-packages/nova/utils.py:183
> 2013-02-11 17:04:01 DEBUG nova.utils
> [req-a789eaef-d7f7-4f41-807a-a7b910471a0b
> c5c6eb3e0ea8456c98acd07100a627af c4f9ddbce4f8444c896722e7c672d945]
> Result was 0 execute
> /usr/lib/python2.7/dist-packages/nova/utils.py:199
> 2013-02-11 17:04:30 DEBUG nova.utils
> [req-a789eaef-d7f7-4f41-807a-a7b910471a0b
> c5c6eb3e0ea8456c98acd07100a627af c4f9ddbce4f8444c896722e7c672d945] Got
> semaphore "compute_resources" for method "update_usage"... inner
> /usr/lib/python2.7/dist-packages/nova/utils.py:721
> 2013-02-11 17:04:30 24056 ERROR nova.openstack.common.rpc.amqp [-]
> Exception during message handling
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> Traceback (most recent call last):
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py",
> line 276, in _process_data
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> rval = self.proxy.dispatch(ctxt, version, method, **args)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py",
> line 145, in dispatch
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> return getattr(proxyobj, method)(ctxt, **kwargs)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/exception.py", line 117, in
> wrapped
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> temp_level, payload)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> self.gen.next()
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/exception.py", line 92, in
> wrapped
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> return f(*args, **kw)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 175,
> in decorated_function
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp     pass
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> self.gen.next()
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 161,
> in decorated_function
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> return function(self, context, *args, **kwargs)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 196,
> in decorated_function
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> kwargs['instance']['uuid'], e, sys.exc_info())
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> self.gen.next()
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 190,
> in decorated_function
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> return function(self, context, *args, **kwargs)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1174,
> in snapshot_instance
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> self.driver.snapshot(context, instance, image_id)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/exception.py", line 117, in
> wrapped
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> temp_level, payload)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> self.gen.next()
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/exception.py", line 92, in
> wrapped
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> return f(*args, **kw)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line
> 882, in snapshot
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp     image_file)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 253, in
> update
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> _reraise_translated_image_exception(image_id)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 251, in
> update
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> image_id, **image_meta)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 138, in
> call
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> return getattr(client.images, method)(*args, **kwargs)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/glanceclient/v1/images.py", line
> 253, in update
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> 'PUT', url, headers=hdrs, body=image_data)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line
> 191, in raw_request
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> return self._http_request(url, method, **kwargs)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp   File
> "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line
> 158, in _http_request
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> raise exc.from_response(resp)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
> Invalid: HTTPBadRequest (HTTP 400)
> 2013-02-11 17:04:30 24056 TRACE nova.openstack.common.rpc.amqp
>
> Thank in advance for your help
> .a.
>
> --
> antonio.s.messina@xxxxxxxxx
> GC3: Grid Computing Competence Center
> http://www.gc3.uzh.ch/
> University of Zurich
> Winterthurerstrasse 190
> CH-8057 Zurich Switzerland



-- 
antonio.s.messina@xxxxxxxxx
GC3: Grid Computing Competence Center
http://www.gc3.uzh.ch/
University of Zurich
Winterthurerstrasse 190
CH-8057 Zurich Switzerland