← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1288144] [NEW] check-tempest-dsvm-full: test_volumes_snapshots fail with SnapshotBuildError

 

Public bug reported:

The log file is here: http://logs.openstack.org/57/62557/29/check/check-
tempest-dsvm-full/ace1c31/logs/testr_results.html.gz

2014-03-05 09:02:21.414 | Traceback (most recent call last):
2014-03-05 09:02:21.414 |   File "tempest/api/volume/test_volumes_snapshots.py", line 85, in test_snapshot_create_with_volume_in_use
2014-03-05 09:02:21.414 |     force=True)
2014-03-05 09:02:21.414 |   File "tempest/api/volume/base.py", line 65, in create_snapshot
2014-03-05 09:02:21.414 |     'available')
2014-03-05 09:02:21.414 |   File "tempest/services/volume/xml/snapshots_client.py", line 138, in wait_for_snapshot_status
2014-03-05 09:02:21.414 |     value = self._get_snapshot_status(snapshot_id)
2014-03-05 09:02:21.414 |   File "tempest/services/volume/xml/snapshots_client.py", line 111, in _get_snapshot_status
2014-03-05 09:02:21.415 |     snapshot_id=snapshot_id)
2014-03-05 09:02:21.415 | SnapshotBuildErrorException: Snapshot a67cb89e-cc78-4e29-85d4-295ba5635e9f failed to build and is in ERROR status
2014-03-05 09:02:21.415 | 
2014-03-05 09:02:21.415 | 
2014-03-05 09:02:21.415 | ======================================================================
2014-03-05 09:02:21.415 | FAIL: process-returncode
2014-03-05 09:02:21.415 | process-returncode
2014-03-05 09:02:21.415 | ----------------------------------------------------------------------
2014-03-05 09:02:21.415 | _StringException: Binary content:
2014-03-05 09:02:21.415 |   traceback (test/plain; charset="utf8")
2014-03-05 09:02:21.415 | 
2014-03-05 09:02:21.415 | 
2014-03-05 09:02:21.416 | ----------------------------------------------------------------------
2014-03-05 09:02:21.416 | Ran 2170 tests in 1995.605s
2014-03-05 09:02:21.416 | 
2014-03-05 09:02:21.416 | FAILED (failures=2, skipped=138)
2014-03-05 09:02:21.416 | ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh (?!.*\\[.*\\bslow\\b.*\\])(^tempest\\.(api|scenario|thirdparty|cli)) --concurrency=4'
2014-03-05 09:02:21.416 | ___________________________________ summary ____________________________________
2014-03-05 09:02:21.417 | ERROR:   full: commands failed
2014-03-05 09:02:21.451 | Checking logs...
2014-03-05 09:02:21.517 | Log File: h-eng
2014-03-05 09:02:21.517 | 2014-03-05 08:43:53.257 30907 ERROR heat.openstack.common.rpc.amqp [req-e630a53b-c9e7-4503-89f9-57c92b2fedf7 None] Exception during message handling
2014-03-05 09:02:21.517 | 
2014-03-05 09:02:21.518 | 2014-03-05 08:43:53.258 30907 ERROR heat.openstack.common.rpc.common [req-e630a53b-c9e7-4503-89f9-57c92b2fedf7 None] Returning exception The Stack (heat-862201009) could not be found. to caller
2014-03-05 09:02:21.518 | 
2014-03-05 09:02:21.518 | 2014-03-05 08:43:53.258 30907 ERROR heat.openstack.common.rpc.common [req-e630a53b-c9e7-4503-89f9-57c92b2fedf7 None] ['Traceback (most recent call last):\n', '  File "/opt/stack/new/heat/heat/openstack/common/rpc/amqp.py", line 462, in _process_data\n    **args)\n', '  File "/opt/stack/new/heat/heat/openstack/common/rpc/dispatcher.py", line 172, in dispatch\n    result = getattr(proxyobj, method)(ctxt, **kwargs)\n', '  File "/opt/stack/new/heat/heat/engine/service.py", line 63, in wrapped\n    return func(self, ctx, *args, **kwargs)\n', '  File "/opt/stack/new/heat/heat/engine/service.py", line 615, in delete_stack\n    st = self._get_stack(cnxt, stack_identity)\n', '  File "/opt/stack/new/heat/heat/engine/service.py", line 270, in _get_stack\n    raise exception.StackNotFound(stack_name=identity.stack_name)\n', 'StackNotFound: The Stack (heat-862201009) could not be found.\n']
2014-03-05 09:02:21.519 | 
2014-03-05 09:02:21.534 | Log File: ceilometer-acompute
2014-03-05 09:02:21.534 | 2014-03-05 08:36:25.085 30132 ERROR ceilometer.compute.pollsters.disk [-] Requested operation is not valid: domain is not running
2014-03-05 09:02:21.534 | 
2014-03-05 09:02:21.648 | Log File: n-net
2014-03-05 09:02:21.648 | *** Not Whitelisted *** 2014-03-05 08:40:53.991 29029 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Instance 8cd15081-db14-435e-85d6-1998dc6a3bc5 could not be found.
2014-03-05 09:02:21.648 | 
2014-03-05 09:02:21.649 | *** Not Whitelisted *** 2014-03-05 08:40:53.994 29029 ERROR oslo.messaging._drivers.common [-] Returning exception Instance 8cd15081-db14-435e-85d6-1998dc6a3bc5 could not be found.
2014-03-05 09:02:21.649 | 
2014-03-05 09:02:21.649 | *** Not Whitelisted *** 2014-03-05 08:40:53.994 29029 ERROR oslo.messaging._drivers.common [-] ['Traceback (most recent call last):\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n    incoming.message))\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/opt/stack/new/nova/nova/network/floating_ips.py", line 119, in allocate_for_instance\n    **kwargs)\n', '  File "/opt/stack/new/nova/nova/network/manager.py", line 516, in allocate_for_instance\n    requested_networks=requested_networks)\n', '  File "/opt/stack/new/nova/nova/network/manager.py", line 216, in _allocate_fixed_ips\n    vpn=vpn, address=address)\n', '  File "/opt/stack/new/nova/nova/network/manager.py", line 895, in allocate_fixed_ip\n    self.quotas.rollback(context, reservations)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/network/manager.py", line 879, in allocate_fixed_ip\n    instance = instance_obj.Instance.get_by_uuid(context, instance_id)\n', '  File "/opt/stack/new/nova/nova/objects/base.py", line 110, in wrapper\n    args, kwargs)\n', '  File "/opt/stack/new/nova/nova/conductor/rpcapi.py", line 483, in object_class_action\n    objver=objver, args=args, kwargs=kwargs)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/client.py", line 150, in call\n    wait_for_reply=True, timeout=timeout)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/transport.py", line 90, in _send\n    timeout=timeout)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/_drivers/amqpdriver.py", line 409, in send\n    return self._send(target, ctxt, message, wait_for_reply, timeout)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/_drivers/amqpdriver.py", line 402, in _send\n    raise result\n', 'InstanceNotFound_Remote: Instance 8cd15081-db14-435e-85d6-1998dc6a3bc5 could not be found.\nTraceback (most recent call last):\n\n  File "/opt/stack/new/nova/nova/conductor/manager.py", line 583, in _object_dispatch\n    return getattr(target, method)(context, *args, **kwargs)\n\n  File "/opt/stack/new/nova/nova/objects/base.py", line 112, in wrapper\n    result = fn(cls, context, *args, **kwargs)\n\n  File "/opt/stack/new/nova/nova/objects/instance.py", line 313, in get_by_uuid\n    use_slave=use_slave)\n\n  File "/opt/stack/new/nova/nova/db/api.py", line 635, in instance_get_by_uuid\n    columns_to_join, use_slave=use_slave)\n\n  File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 128, in wrapper\n    return f(*args, **kwargs)\n\n  File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1649, in instance_get_by_uuid\n    columns_to_join=columns_to_join, use_slave=use_slave)\n\n  File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1661, in _instance_get_by_uuid\n    raise exception.InstanceNotFound(instance_id=uuid)\n\nInstanceNotFound: Instance 8cd15081-db14-435e-85d6-1998dc6a3bc5 could not be found.\n\n']
2014-03-05 09:02:21.649 | 
2014-03-05 09:02:21.790 | Log File: c-vol
2014-03-05 09:02:21.790 | *** Not Whitelisted *** 2014-03-05 08:57:02.731 29953 ERROR cinder.openstack.common.rpc.amqp [req-ae37cf0a-dcf4-455c-8f2b-a3242ac3c8c3 None] Exception during message handling
2014-03-05 09:02:21.790 | 
2014-03-05 09:02:21.905 | Log File: n-cpu
2014-03-05 09:02:21.905 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] Traceback (most recent call last):
2014-03-05 09:02:21.905 | 
2014-03-05 09:02:21.906 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/opt/stack/new/nova/nova/compute/manager.py", line 310, in decorated_function
2014-03-05 09:02:21.906 | 
2014-03-05 09:02:21.907 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     *args, **kwargs)
2014-03-05 09:02:21.907 | 
2014-03-05 09:02:21.907 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2481, in snapshot_instance
2014-03-05 09:02:21.907 | 
2014-03-05 09:02:21.907 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     task_states.IMAGE_SNAPSHOT)
2014-03-05 09:02:21.908 | 
2014-03-05 09:02:21.908 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2512, in _snapshot_instance
2014-03-05 09:02:21.909 | 
2014-03-05 09:02:21.909 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     update_task_state)
2014-03-05 09:02:21.909 | 
2014-03-05 09:02:21.910 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1520, in snapshot
2014-03-05 09:02:21.910 | 
2014-03-05 09:02:21.910 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     virt_dom.managedSave(0)
2014-03-05 09:02:21.910 | 
2014-03-05 09:02:21.910 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
2014-03-05 09:02:21.910 | 
2014-03-05 09:02:21.911 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-03-05 09:02:21.911 | 
2014-03-05 09:02:21.911 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call
2014-03-05 09:02:21.911 | 
2014-03-05 09:02:21.911 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     rv = execute(f,*args,**kwargs)
2014-03-05 09:02:21.911 | 
2014-03-05 09:02:21.935 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
2014-03-05 09:02:21.935 | 
2014-03-05 09:02:21.935 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     rv = meth(*args,**kwargs)
2014-03-05 09:02:21.935 | 
2014-03-05 09:02:21.936 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 696, in managedSave
2014-03-05 09:02:21.936 | 
2014-03-05 09:02:21.936 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     if ret == -1: raise libvirtError ('virDomainManagedSave() failed', dom=self)
2014-03-05 09:02:21.936 | 
2014-03-05 09:02:21.936 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] libvirtError: Unable to write to monitor: Broken pipe
2014-03-05 09:02:21.936 | 
2014-03-05 09:02:21.937 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] 
2014-03-05 09:02:21.937 | 
2014-03-05 09:02:21.937 | 2014-03-05 08:34:04.183 ERROR nova.compute.manager [req-5b6f9f17-7c5c-406e-8a3b-6c7573c9ef90 ImagesTestJSON-1946939531 ImagesTestJSON-361842792] [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] Error while trying to clean up image c1b2f117-757e-42ff-9ec2-ebc44f7ad162
2014-03-05 09:02:21.937 | 
2014-03-05 09:02:21.937 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] Traceback (most recent call last):
2014-03-05 09:02:21.937 | 
2014-03-05 09:02:21.938 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/opt/stack/new/nova/nova/compute/manager.py", line 317, in decorated_function
2014-03-05 09:02:21.938 | 
2014-03-05 09:02:21.938 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     image_service.delete(context, image_id)
2014-03-05 09:02:21.938 | 
2014-03-05 09:02:21.938 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/opt/stack/new/nova/nova/image/glance.py", line 407, in delete
2014-03-05 09:02:21.938 | 
2014-03-05 09:02:21.939 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     raise exception.ImageNotFound(image_id=image_id)
2014-03-05 09:02:21.939 | 
2014-03-05 09:02:21.939 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] ImageNotFound: Image c1b2f117-757e-42ff-9ec2-ebc44f7ad162 could not be found.
2014-03-05 09:02:21.939 | 
2014-03-05 09:02:21.939 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] 
2014-03-05 09:02:21.939 | 
2014-03-05 09:02:21.939 | *** Not Whitelisted *** 2014-03-05 08:34:04.281 28941 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Unable to write to monitor: Broken pipe
2014-03-05 09:02:21.940 | 
2014-03-05 09:02:21.940 | *** Not Whitelisted *** 2014-03-05 08:34:04.282 28941 ERROR oslo.messaging._drivers.common [-] Returning exception Unable to write to monitor: Broken pipe to caller
2014-03-05 09:02:21.940 | 
2014-03-05 09:02:21.940 | *** Not Whitelisted *** 2014-03-05 08:34:04.283 28941 ERROR oslo.messaging._drivers.common [-] ['Traceback (most recent call last):\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n    incoming.message))\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/opt/stack/new/nova/nova/exception.py", line 88, in wrapped\n    payload)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/exception.py", line 71, in wrapped\n    return f(self, context, *args, **kw)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 244, in decorated_function\n    pass\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 230, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 272, in decorated_function\n    e, sys.exc_info())\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 259, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 320, in decorated_function\n    % image_id, instance=instance)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 310, in decorated_function\n    *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 2481, in snapshot_instance\n    task_states.IMAGE_SNAPSHOT)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 2512, in _snapshot_instance\n    update_task_state)\n', '  File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1520, in snapshot\n    virt_dom.managedSave(0)\n', '  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit\n    result = proxy_call(self._autowrap, f, *args, **kwargs)\n', '  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call\n    rv = execute(f,*args,**kwargs)\n', '  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker\n    rv = meth(*args,**kwargs)\n', '  File "/usr/lib/python2.7/dist-packages/libvirt.py", line 696, in managedSave\n    if ret == -1: raise libvirtError (\'virDomainManagedSave() failed\', dom=self)\n', 'libvirtError: Unable to write to monitor: Broken pipe\n']
2014-03-05 09:02:21.940 | 
2014-03-05 09:02:21.940 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] Traceback (most recent call last):
2014-03-05 09:02:21.941 | 
2014-03-05 09:02:21.941 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]   File "/opt/stack/new/nova/nova/compute/manager.py", line 310, in decorated_function
2014-03-05 09:02:21.941 | 
2014-03-05 09:02:21.941 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]     *args, **kwargs)
2014-03-05 09:02:21.941 | 
2014-03-05 09:02:21.941 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2481, in snapshot_instance
2014-03-05 09:02:21.941 | 
2014-03-05 09:02:21.942 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]     task_states.IMAGE_SNAPSHOT)
2014-03-05 09:02:21.942 | 
2014-03-05 09:02:21.942 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2512, in _snapshot_instance
2014-03-05 09:02:21.942 | 
2014-03-05 09:02:21.942 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]     update_task_state)
2014-03-05 09:02:21.942 | 
2014-03-05 09:02:21.943 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1455, in snapshot
2014-03-05 09:02:21.943 | 
2014-03-05 09:02:21.943 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]     raise exception.InstanceNotRunning(instance_id=instance['uuid'])
2014-03-05 09:02:21.943 | 
2014-03-05 09:02:21.943 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] InstanceNotRunning: Instance fe3019e7-efda-4e00-b58b-9e9f63103632 is not running.
2014-03-05 09:02:21.943 | 
2014-03-05 09:02:21.944 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] 
2014-03-05 09:02:21.944 | 
2014-03-05 09:02:21.944 | 2014-03-05 08:34:22.509 ERROR nova.compute.manager [req-eaa57e82-a3ac-4e26-83c3-ac2914ee611e ImagesOneServerTestJSON-391714282 ImagesOneServerTestJSON-606186356] [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] Error while trying to clean up image 497e7e9f-7924-4499-ad6b-cb272af2069d
2014-03-05 09:02:21.944 | 
2014-03-05 09:02:21.944 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] Traceback (most recent call last):
2014-03-05 09:02:21.944 | 
2014-03-05 09:02:21.945 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]   File "/opt/stack/new/nova/nova/compute/manager.py", line 317, in decorated_function
2014-03-05 09:02:21.945 | 
2014-03-05 09:02:21.945 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]     image_service.delete(context, image_id)
2014-03-05 09:02:21.945 | 
2014-03-05 09:02:21.945 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]   File "/opt/stack/new/nova/nova/image/glance.py", line 407, in delete
2014-03-05 09:02:21.945 | 
2014-03-05 09:02:21.946 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]     raise exception.ImageNotFound(image_id=image_id)
2014-03-05 09:02:21.946 | 
2014-03-05 09:02:21.946 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] ImageNotFound: Image 497e7e9f-7924-4499-ad6b-cb272af2069d could not be found.
2014-03-05 09:02:21.946 | 
2014-03-05 09:02:21.946 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] 
2014-03-05 09:02:21.946 | 
2014-03-05 09:02:21.947 | *** Not Whitelisted *** 2014-03-05 08:34:22.870 28941 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Instance fe3019e7-efda-4e00-b58b-9e9f63103632 is not running.
2014-03-05 09:02:21.947 | 
2014-03-05 09:02:21.947 | *** Not Whitelisted *** 2014-03-05 08:34:22.872 28941 ERROR oslo.messaging._drivers.common [-] Returning exception Instance fe3019e7-efda-4e00-b58b-9e9f63103632 is not running. to caller
2014-03-05 09:02:21.947 | 
2014-03-05 09:02:21.947 | *** Not Whitelisted *** 2014-03-05 08:34:22.872 28941 ERROR oslo.messaging._drivers.common [-] ['Traceback (most recent call last):\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n    incoming.message))\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/opt/stack/new/nova/nova/exception.py", line 88, in wrapped\n    payload)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/exception.py", line 71, in wrapped\n    return f(self, context, *args, **kw)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 244, in decorated_function\n    pass\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 230, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 272, in decorated_function\n    e, sys.exc_info())\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 259, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 320, in decorated_function\n    % image_id, instance=instance)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 310, in decorated_function\n    *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 2481, in snapshot_instance\n    task_states.IMAGE_SNAPSHOT)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 2512, in _snapshot_instance\n    update_task_state)\n', '  File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1455, in snapshot\n    raise exception.InstanceNotRunning(instance_id=instance[\'uuid\'])\n', 'InstanceNotRunning: Instance fe3019e7-efda-4e00-b58b-9e9f63103632 is not running.\n']
2014-03-05 09:02:21.948 | 
2014-03-05 09:02:21.968 | 2014-03-05 08:38:23.761 28941 ERROR nova.virt.libvirt.driver [-] [instance: 5fc72019-fa1f-47fd-9441-dba16e34da89] During wait destroy, instance disappeared.
2014-03-05 09:02:21.968 | 
2014-03-05 09:02:21.975 | 2014-03-05 08:38:44.399 28941 ERROR nova.virt.libvirt.driver [-] [instance: abadeb0e-a22d-4b6e-b4d2-f94fb6e22b88] During wait destroy, instance disappeared.
2014-03-05 09:02:21.975 | 
2014-03-05 09:02:21.999 | 2014-03-05 08:40:53.999 28941 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
2014-03-05 09:02:21.999 | 
2014-03-05 09:02:22.087 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] Traceback (most recent call last):
2014-03-05 09:02:22.087 | 
2014-03-05 09:02:22.088 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]   File "/opt/stack/new/nova/nova/compute/manager.py", line 310, in decorated_function
2014-03-05 09:02:22.088 | 
2014-03-05 09:02:22.088 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]     *args, **kwargs)
2014-03-05 09:02:22.088 | 
2014-03-05 09:02:22.089 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2481, in snapshot_instance
2014-03-05 09:02:22.089 | 
2014-03-05 09:02:22.089 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]     task_states.IMAGE_SNAPSHOT)
2014-03-05 09:02:22.089 | 
2014-03-05 09:02:22.089 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2512, in _snapshot_instance
2014-03-05 09:02:22.089 | 
2014-03-05 09:02:22.090 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]     update_task_state)
2014-03-05 09:02:22.090 | 
2014-03-05 09:02:22.090 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1455, in snapshot
2014-03-05 09:02:22.090 | 
2014-03-05 09:02:22.090 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]     raise exception.InstanceNotRunning(instance_id=instance['uuid'])
2014-03-05 09:02:22.090 | 
2014-03-05 09:02:22.091 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] InstanceNotRunning: Instance 2576443e-3e73-42e5-b33f-c6f427ffaf65 is not running.
2014-03-05 09:02:22.091 | 
2014-03-05 09:02:22.091 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] 
2014-03-05 09:02:22.091 | 
2014-03-05 09:02:22.126 | 2014-03-05 08:47:18.032 ERROR nova.compute.manager [req-54c023e1-e30d-458c-905a-8cd7df35df0a ImagesOneServerV3Test-1071173160 ImagesOneServerV3Test-341425521] [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] Error while trying to clean up image 73ebe813-c02b-4204-b8cd-651e225871e3
2014-03-05 09:02:22.126 | 
2014-03-05 09:02:22.126 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] Traceback (most recent call last):
2014-03-05 09:02:22.127 | 
2014-03-05 09:02:22.127 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]   File "/opt/stack/new/nova/nova/compute/manager.py", line 317, in decorated_function
2014-03-05 09:02:22.127 | 
2014-03-05 09:02:22.127 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]     image_service.delete(context, image_id)
2014-03-05 09:02:22.127 | 
2014-03-05 09:02:22.128 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]   File "/opt/stack/new/nova/nova/image/glance.py", line 407, in delete
2014-03-05 09:02:22.128 | 
2014-03-05 09:02:22.128 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]     raise exception.ImageNotFound(image_id=image_id)
2014-03-05 09:02:22.128 | 
2014-03-05 09:02:22.128 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] ImageNotFound: Image 73ebe813-c02b-4204-b8cd-651e225871e3 could not be found.
2014-03-05 09:02:22.129 | 
2014-03-05 09:02:22.129 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] 
2014-03-05 09:02:22.129 | 
2014-03-05 09:02:22.129 | *** Not Whitelisted *** 2014-03-05 08:47:18.188 28941 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Instance 2576443e-3e73-42e5-b33f-c6f427ffaf65 is not running.
2014-03-05 09:02:22.129 | 
2014-03-05 09:02:22.130 | *** Not Whitelisted *** 2014-03-05 08:47:18.189 28941 ERROR oslo.messaging._drivers.common [-] Returning exception Instance 2576443e-3e73-42e5-b33f-c6f427ffaf65 is not running. to caller
2014-03-05 09:02:22.130 | 
2014-03-05 09:02:22.130 | *** Not Whitelisted *** 2014-03-05 08:47:18.189 28941 ERROR oslo.messaging._drivers.common [-] ['Traceback (most recent call last):\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n    incoming.message))\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/opt/stack/new/nova/nova/exception.py", line 88, in wrapped\n    payload)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/exception.py", line 71, in wrapped\n    return f(self, context, *args, **kw)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 244, in decorated_function\n    pass\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 230, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 272, in decorated_function\n    e, sys.exc_info())\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 259, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 320, in decorated_function\n    % image_id, instance=instance)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 310, in decorated_function\n    *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 2481, in snapshot_instance\n    task_states.IMAGE_SNAPSHOT)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 2512, in _snapshot_instance\n    update_task_state)\n', '  File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1455, in snapshot\n    raise exception.InstanceNotRunning(instance_id=instance[\'uuid\'])\n', 'InstanceNotRunning: Instance 2576443e-3e73-42e5-b33f-c6f427ffaf65 is not running.\n']
2014-03-05 09:02:22.130 | 
2014-03-05 09:02:22.131 | 2014-03-05 08:50:32.939 28941 ERROR nova.virt.libvirt.driver [-] [instance: 23351a8f-3658-45cc-ac21-546faa497135] During wait destroy, instance disappeared.
2014-03-05 09:02:22.132 | 
2014-03-05 09:02:23.400 | Log File: ceilometer-alarm-evaluator
2014-03-05 09:02:23.400 | 2014-03-05 08:26:16.189 30266 ERROR ceilometer.alarm.service [-] alarm evaluation cycle failed
2014-03-05 09:02:23.400 | 
2014-03-05 09:02:23.680 | Log File: n-api
2014-03-05 09:02:23.680 | 2014-03-05 08:44:10.826 ERROR nova.api.openstack [req-b26e1663-8d90-4942-9909-e3491f4257eb ServersNegativeTestJSON-1261521225 ServersNegativeTestJSON-139481132] Caught error: Instance 3d9f406f-b4d0-41b1-b83a-ecf0330a99cc could not be found.
2014-03-05 09:02:23.681 | 
2014-03-05 09:02:23.683 | 2014-03-05 08:44:15.568 ERROR nova.api.openstack [req-4eb3cd32-f1d7-4909-95f2-4bb43e76afdf ServersNegativeTestJSON-1261521225 ServersNegativeTestJSON-139481132] Caught error: Instance 8374d83b-c35d-4670-a0b1-6bd4168ee026 could not be found.
2014-03-05 09:02:23.683 | 
2014-03-05 09:02:23.697 | 2014-03-05 08:45:16.125 ERROR nova.api.openstack [req-933bd333-6de4-4387-8610-3f59ce4246dc VirtualInterfacesNegativeTestJSON-1503600377 VirtualInterfacesNegativeTestJSON-1464645285] Caught error: Instance a7c4748e-34d2-4dca-9ca9-7df653ad25a2 could not be found.
2014-03-05 09:02:23.698 | 
2014-03-05 09:02:23.720 | 2014-03-05 08:46:22.756 ERROR nova.api.openstack [req-f38a2dfc-a841-466f-8e6e-f95e4be75736 ServersNegativeTestXML-544157769 ServersNegativeTestXML-1067209753] Caught error: Instance e17a869f-ecde-4770-9e60-89e4916a7d92 could not be found.
2014-03-05 09:02:23.720 | 
2014-03-05 09:02:23.723 | 2014-03-05 08:46:28.970 ERROR nova.api.openstack [req-f12c6218-b13d-42cf-9620-d31deb8f309e ServersNegativeTestXML-544157769 ServersNegativeTestXML-1067209753] Caught error: Instance f376a419-2751-49cb-9113-bb343d38af85 could not be found.
2014-03-05 09:02:23.723 | 
2014-03-05 09:02:23.730 | 2014-03-05 08:47:04.340 ERROR nova.api.openstack [req-9e8a00a7-45ce-41cb-a1cb-cb88a177fc7a ServersAdminV3Test-316812114 ServersAdminV3Test-1656858922] Caught error: Instance ec774d4e-8cd3-4dc2-b90f-8ad1c01548de could not be found.
2014-03-05 09:02:23.730 | 
2014-03-05 09:02:23.732 | 2014-03-05 08:47:10.502 ERROR nova.api.openstack [req-5e512558-f188-42fb-a739-f25a5495429c VirtualInterfacesNegativeTestXML-2023142622 VirtualInterfacesNegativeTestXML-1968584697] Caught error: Instance 6318125a-920d-4dd8-b31d-5e227c006463 could not be found.
2014-03-05 09:02:23.732 | 
2014-03-05 09:02:23.798 | 2014-03-05 08:52:05.139 ERROR nova.api.openstack [req-74aa8088-08c4-4a2a-adf5-c1afe2e90cfc ServersV3Test-1818645145 ServersV3Test-161538606] Caught error: Instance 333c06ea-430a-4ee1-90ca-132e61604ef3 could not be found.
2014-03-05 09:02:23.798 | 
2014-03-05 09:02:24.441 | Log File: g-api
2014-03-05 09:02:24.442 | 2014-03-05 08:25:55.309 27572 ERROR swiftclient [4b4be3d1-eaf4-4f68-ab4d-c3091080cf81 1425971ac0b44225b4cf6e0649337aeb 3063ee5211114763b3ec1046563b18b5 - - -] Container HEAD failed: http://127.0.0.1:8080:8080/v1/AUTH_08965a364f3646febdcc3ea1566cb865/glance 404 Not Found
2014-03-05 09:02:24.442 | 
2014-03-05 09:02:24.802 | Log File: c-api
2014-03-05 09:02:24.802 | 2014-03-05 08:54:35.679 30090 ERROR cinder.volume.api [req-1240d8f1-435f-4f74-9d91-08e9f6e8e24d None] Volume status must be available to reserve
2014-03-05 09:02:24.802 | 
2014-03-05 09:02:24.811 | 2014-03-05 08:54:48.380 30090 ERROR cinder.volume.api [req-4b69fd70-34fd-4954-8f24-3ab55d10cd9c None] Volume status must be available to reserve
2014-03-05 09:02:24.811 | 
2014-03-05 09:02:24.851 | Logs have errors
2014-03-05 09:02:24.851 | FAILED
2014-03-05 09:02:24.859 | ERROR: gate_hook failed

** Affects: openstack-ci
     Importance: Undecided
         Status: New

** Project changed: nova => openstack-ci

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1288144

Title:
  check-tempest-dsvm-full: test_volumes_snapshots fail with
  SnapshotBuildError

Status in OpenStack Core Infrastructure:
  New

Bug description:
  The log file is here: http://logs.openstack.org/57/62557/29/check
  /check-tempest-dsvm-full/ace1c31/logs/testr_results.html.gz

  2014-03-05 09:02:21.414 | Traceback (most recent call last):
  2014-03-05 09:02:21.414 |   File "tempest/api/volume/test_volumes_snapshots.py", line 85, in test_snapshot_create_with_volume_in_use
  2014-03-05 09:02:21.414 |     force=True)
  2014-03-05 09:02:21.414 |   File "tempest/api/volume/base.py", line 65, in create_snapshot
  2014-03-05 09:02:21.414 |     'available')
  2014-03-05 09:02:21.414 |   File "tempest/services/volume/xml/snapshots_client.py", line 138, in wait_for_snapshot_status
  2014-03-05 09:02:21.414 |     value = self._get_snapshot_status(snapshot_id)
  2014-03-05 09:02:21.414 |   File "tempest/services/volume/xml/snapshots_client.py", line 111, in _get_snapshot_status
  2014-03-05 09:02:21.415 |     snapshot_id=snapshot_id)
  2014-03-05 09:02:21.415 | SnapshotBuildErrorException: Snapshot a67cb89e-cc78-4e29-85d4-295ba5635e9f failed to build and is in ERROR status
  2014-03-05 09:02:21.415 | 
  2014-03-05 09:02:21.415 | 
  2014-03-05 09:02:21.415 | ======================================================================
  2014-03-05 09:02:21.415 | FAIL: process-returncode
  2014-03-05 09:02:21.415 | process-returncode
  2014-03-05 09:02:21.415 | ----------------------------------------------------------------------
  2014-03-05 09:02:21.415 | _StringException: Binary content:
  2014-03-05 09:02:21.415 |   traceback (test/plain; charset="utf8")
  2014-03-05 09:02:21.415 | 
  2014-03-05 09:02:21.415 | 
  2014-03-05 09:02:21.416 | ----------------------------------------------------------------------
  2014-03-05 09:02:21.416 | Ran 2170 tests in 1995.605s
  2014-03-05 09:02:21.416 | 
  2014-03-05 09:02:21.416 | FAILED (failures=2, skipped=138)
  2014-03-05 09:02:21.416 | ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh (?!.*\\[.*\\bslow\\b.*\\])(^tempest\\.(api|scenario|thirdparty|cli)) --concurrency=4'
  2014-03-05 09:02:21.416 | ___________________________________ summary ____________________________________
  2014-03-05 09:02:21.417 | ERROR:   full: commands failed
  2014-03-05 09:02:21.451 | Checking logs...
  2014-03-05 09:02:21.517 | Log File: h-eng
  2014-03-05 09:02:21.517 | 2014-03-05 08:43:53.257 30907 ERROR heat.openstack.common.rpc.amqp [req-e630a53b-c9e7-4503-89f9-57c92b2fedf7 None] Exception during message handling
  2014-03-05 09:02:21.517 | 
  2014-03-05 09:02:21.518 | 2014-03-05 08:43:53.258 30907 ERROR heat.openstack.common.rpc.common [req-e630a53b-c9e7-4503-89f9-57c92b2fedf7 None] Returning exception The Stack (heat-862201009) could not be found. to caller
  2014-03-05 09:02:21.518 | 
  2014-03-05 09:02:21.518 | 2014-03-05 08:43:53.258 30907 ERROR heat.openstack.common.rpc.common [req-e630a53b-c9e7-4503-89f9-57c92b2fedf7 None] ['Traceback (most recent call last):\n', '  File "/opt/stack/new/heat/heat/openstack/common/rpc/amqp.py", line 462, in _process_data\n    **args)\n', '  File "/opt/stack/new/heat/heat/openstack/common/rpc/dispatcher.py", line 172, in dispatch\n    result = getattr(proxyobj, method)(ctxt, **kwargs)\n', '  File "/opt/stack/new/heat/heat/engine/service.py", line 63, in wrapped\n    return func(self, ctx, *args, **kwargs)\n', '  File "/opt/stack/new/heat/heat/engine/service.py", line 615, in delete_stack\n    st = self._get_stack(cnxt, stack_identity)\n', '  File "/opt/stack/new/heat/heat/engine/service.py", line 270, in _get_stack\n    raise exception.StackNotFound(stack_name=identity.stack_name)\n', 'StackNotFound: The Stack (heat-862201009) could not be found.\n']
  2014-03-05 09:02:21.519 | 
  2014-03-05 09:02:21.534 | Log File: ceilometer-acompute
  2014-03-05 09:02:21.534 | 2014-03-05 08:36:25.085 30132 ERROR ceilometer.compute.pollsters.disk [-] Requested operation is not valid: domain is not running
  2014-03-05 09:02:21.534 | 
  2014-03-05 09:02:21.648 | Log File: n-net
  2014-03-05 09:02:21.648 | *** Not Whitelisted *** 2014-03-05 08:40:53.991 29029 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Instance 8cd15081-db14-435e-85d6-1998dc6a3bc5 could not be found.
  2014-03-05 09:02:21.648 | 
  2014-03-05 09:02:21.649 | *** Not Whitelisted *** 2014-03-05 08:40:53.994 29029 ERROR oslo.messaging._drivers.common [-] Returning exception Instance 8cd15081-db14-435e-85d6-1998dc6a3bc5 could not be found.
  2014-03-05 09:02:21.649 | 
  2014-03-05 09:02:21.649 | *** Not Whitelisted *** 2014-03-05 08:40:53.994 29029 ERROR oslo.messaging._drivers.common [-] ['Traceback (most recent call last):\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n    incoming.message))\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/opt/stack/new/nova/nova/network/floating_ips.py", line 119, in allocate_for_instance\n    **kwargs)\n', '  File "/opt/stack/new/nova/nova/network/manager.py", line 516, in allocate_for_instance\n    requested_networks=requested_networks)\n', '  File "/opt/stack/new/nova/nova/network/manager.py", line 216, in _allocate_fixed_ips\n    vpn=vpn, address=address)\n', '  File "/opt/stack/new/nova/nova/network/manager.py", line 895, in allocate_fixed_ip\n    self.quotas.rollback(context, reservations)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/network/manager.py", line 879, in allocate_fixed_ip\n    instance = instance_obj.Instance.get_by_uuid(context, instance_id)\n', '  File "/opt/stack/new/nova/nova/objects/base.py", line 110, in wrapper\n    args, kwargs)\n', '  File "/opt/stack/new/nova/nova/conductor/rpcapi.py", line 483, in object_class_action\n    objver=objver, args=args, kwargs=kwargs)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/client.py", line 150, in call\n    wait_for_reply=True, timeout=timeout)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/transport.py", line 90, in _send\n    timeout=timeout)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/_drivers/amqpdriver.py", line 409, in send\n    return self._send(target, ctxt, message, wait_for_reply, timeout)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/_drivers/amqpdriver.py", line 402, in _send\n    raise result\n', 'InstanceNotFound_Remote: Instance 8cd15081-db14-435e-85d6-1998dc6a3bc5 could not be found.\nTraceback (most recent call last):\n\n  File "/opt/stack/new/nova/nova/conductor/manager.py", line 583, in _object_dispatch\n    return getattr(target, method)(context, *args, **kwargs)\n\n  File "/opt/stack/new/nova/nova/objects/base.py", line 112, in wrapper\n    result = fn(cls, context, *args, **kwargs)\n\n  File "/opt/stack/new/nova/nova/objects/instance.py", line 313, in get_by_uuid\n    use_slave=use_slave)\n\n  File "/opt/stack/new/nova/nova/db/api.py", line 635, in instance_get_by_uuid\n    columns_to_join, use_slave=use_slave)\n\n  File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 128, in wrapper\n    return f(*args, **kwargs)\n\n  File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1649, in instance_get_by_uuid\n    columns_to_join=columns_to_join, use_slave=use_slave)\n\n  File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1661, in _instance_get_by_uuid\n    raise exception.InstanceNotFound(instance_id=uuid)\n\nInstanceNotFound: Instance 8cd15081-db14-435e-85d6-1998dc6a3bc5 could not be found.\n\n']
  2014-03-05 09:02:21.649 | 
  2014-03-05 09:02:21.790 | Log File: c-vol
  2014-03-05 09:02:21.790 | *** Not Whitelisted *** 2014-03-05 08:57:02.731 29953 ERROR cinder.openstack.common.rpc.amqp [req-ae37cf0a-dcf4-455c-8f2b-a3242ac3c8c3 None] Exception during message handling
  2014-03-05 09:02:21.790 | 
  2014-03-05 09:02:21.905 | Log File: n-cpu
  2014-03-05 09:02:21.905 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] Traceback (most recent call last):
  2014-03-05 09:02:21.905 | 
  2014-03-05 09:02:21.906 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/opt/stack/new/nova/nova/compute/manager.py", line 310, in decorated_function
  2014-03-05 09:02:21.906 | 
  2014-03-05 09:02:21.907 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     *args, **kwargs)
  2014-03-05 09:02:21.907 | 
  2014-03-05 09:02:21.907 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2481, in snapshot_instance
  2014-03-05 09:02:21.907 | 
  2014-03-05 09:02:21.907 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     task_states.IMAGE_SNAPSHOT)
  2014-03-05 09:02:21.908 | 
  2014-03-05 09:02:21.908 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2512, in _snapshot_instance
  2014-03-05 09:02:21.909 | 
  2014-03-05 09:02:21.909 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     update_task_state)
  2014-03-05 09:02:21.909 | 
  2014-03-05 09:02:21.910 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1520, in snapshot
  2014-03-05 09:02:21.910 | 
  2014-03-05 09:02:21.910 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     virt_dom.managedSave(0)
  2014-03-05 09:02:21.910 | 
  2014-03-05 09:02:21.910 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
  2014-03-05 09:02:21.910 | 
  2014-03-05 09:02:21.911 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     result = proxy_call(self._autowrap, f, *args, **kwargs)
  2014-03-05 09:02:21.911 | 
  2014-03-05 09:02:21.911 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call
  2014-03-05 09:02:21.911 | 
  2014-03-05 09:02:21.911 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     rv = execute(f,*args,**kwargs)
  2014-03-05 09:02:21.911 | 
  2014-03-05 09:02:21.935 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
  2014-03-05 09:02:21.935 | 
  2014-03-05 09:02:21.935 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     rv = meth(*args,**kwargs)
  2014-03-05 09:02:21.935 | 
  2014-03-05 09:02:21.936 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 696, in managedSave
  2014-03-05 09:02:21.936 | 
  2014-03-05 09:02:21.936 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     if ret == -1: raise libvirtError ('virDomainManagedSave() failed', dom=self)
  2014-03-05 09:02:21.936 | 
  2014-03-05 09:02:21.936 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] libvirtError: Unable to write to monitor: Broken pipe
  2014-03-05 09:02:21.936 | 
  2014-03-05 09:02:21.937 | *** Not Whitelisted *** 2014-03-05 08:34:04.112 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] 
  2014-03-05 09:02:21.937 | 
  2014-03-05 09:02:21.937 | 2014-03-05 08:34:04.183 ERROR nova.compute.manager [req-5b6f9f17-7c5c-406e-8a3b-6c7573c9ef90 ImagesTestJSON-1946939531 ImagesTestJSON-361842792] [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] Error while trying to clean up image c1b2f117-757e-42ff-9ec2-ebc44f7ad162
  2014-03-05 09:02:21.937 | 
  2014-03-05 09:02:21.937 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] Traceback (most recent call last):
  2014-03-05 09:02:21.937 | 
  2014-03-05 09:02:21.938 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/opt/stack/new/nova/nova/compute/manager.py", line 317, in decorated_function
  2014-03-05 09:02:21.938 | 
  2014-03-05 09:02:21.938 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     image_service.delete(context, image_id)
  2014-03-05 09:02:21.938 | 
  2014-03-05 09:02:21.938 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]   File "/opt/stack/new/nova/nova/image/glance.py", line 407, in delete
  2014-03-05 09:02:21.938 | 
  2014-03-05 09:02:21.939 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb]     raise exception.ImageNotFound(image_id=image_id)
  2014-03-05 09:02:21.939 | 
  2014-03-05 09:02:21.939 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] ImageNotFound: Image c1b2f117-757e-42ff-9ec2-ebc44f7ad162 could not be found.
  2014-03-05 09:02:21.939 | 
  2014-03-05 09:02:21.939 | *** Not Whitelisted *** 2014-03-05 08:34:04.183 28941 TRACE nova.compute.manager [instance: af5adb07-4ad7-4547-855a-1a5b11ff34bb] 
  2014-03-05 09:02:21.939 | 
  2014-03-05 09:02:21.939 | *** Not Whitelisted *** 2014-03-05 08:34:04.281 28941 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Unable to write to monitor: Broken pipe
  2014-03-05 09:02:21.940 | 
  2014-03-05 09:02:21.940 | *** Not Whitelisted *** 2014-03-05 08:34:04.282 28941 ERROR oslo.messaging._drivers.common [-] Returning exception Unable to write to monitor: Broken pipe to caller
  2014-03-05 09:02:21.940 | 
  2014-03-05 09:02:21.940 | *** Not Whitelisted *** 2014-03-05 08:34:04.283 28941 ERROR oslo.messaging._drivers.common [-] ['Traceback (most recent call last):\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n    incoming.message))\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/opt/stack/new/nova/nova/exception.py", line 88, in wrapped\n    payload)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/exception.py", line 71, in wrapped\n    return f(self, context, *args, **kw)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 244, in decorated_function\n    pass\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 230, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 272, in decorated_function\n    e, sys.exc_info())\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 259, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 320, in decorated_function\n    % image_id, instance=instance)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 310, in decorated_function\n    *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 2481, in snapshot_instance\n    task_states.IMAGE_SNAPSHOT)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 2512, in _snapshot_instance\n    update_task_state)\n', '  File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1520, in snapshot\n    virt_dom.managedSave(0)\n', '  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit\n    result = proxy_call(self._autowrap, f, *args, **kwargs)\n', '  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call\n    rv = execute(f,*args,**kwargs)\n', '  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker\n    rv = meth(*args,**kwargs)\n', '  File "/usr/lib/python2.7/dist-packages/libvirt.py", line 696, in managedSave\n    if ret == -1: raise libvirtError (\'virDomainManagedSave() failed\', dom=self)\n', 'libvirtError: Unable to write to monitor: Broken pipe\n']
  2014-03-05 09:02:21.940 | 
  2014-03-05 09:02:21.940 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] Traceback (most recent call last):
  2014-03-05 09:02:21.941 | 
  2014-03-05 09:02:21.941 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]   File "/opt/stack/new/nova/nova/compute/manager.py", line 310, in decorated_function
  2014-03-05 09:02:21.941 | 
  2014-03-05 09:02:21.941 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]     *args, **kwargs)
  2014-03-05 09:02:21.941 | 
  2014-03-05 09:02:21.941 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2481, in snapshot_instance
  2014-03-05 09:02:21.941 | 
  2014-03-05 09:02:21.942 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]     task_states.IMAGE_SNAPSHOT)
  2014-03-05 09:02:21.942 | 
  2014-03-05 09:02:21.942 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2512, in _snapshot_instance
  2014-03-05 09:02:21.942 | 
  2014-03-05 09:02:21.942 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]     update_task_state)
  2014-03-05 09:02:21.942 | 
  2014-03-05 09:02:21.943 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1455, in snapshot
  2014-03-05 09:02:21.943 | 
  2014-03-05 09:02:21.943 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]     raise exception.InstanceNotRunning(instance_id=instance['uuid'])
  2014-03-05 09:02:21.943 | 
  2014-03-05 09:02:21.943 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] InstanceNotRunning: Instance fe3019e7-efda-4e00-b58b-9e9f63103632 is not running.
  2014-03-05 09:02:21.943 | 
  2014-03-05 09:02:21.944 | *** Not Whitelisted *** 2014-03-05 08:34:22.438 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] 
  2014-03-05 09:02:21.944 | 
  2014-03-05 09:02:21.944 | 2014-03-05 08:34:22.509 ERROR nova.compute.manager [req-eaa57e82-a3ac-4e26-83c3-ac2914ee611e ImagesOneServerTestJSON-391714282 ImagesOneServerTestJSON-606186356] [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] Error while trying to clean up image 497e7e9f-7924-4499-ad6b-cb272af2069d
  2014-03-05 09:02:21.944 | 
  2014-03-05 09:02:21.944 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] Traceback (most recent call last):
  2014-03-05 09:02:21.944 | 
  2014-03-05 09:02:21.945 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]   File "/opt/stack/new/nova/nova/compute/manager.py", line 317, in decorated_function
  2014-03-05 09:02:21.945 | 
  2014-03-05 09:02:21.945 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]     image_service.delete(context, image_id)
  2014-03-05 09:02:21.945 | 
  2014-03-05 09:02:21.945 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]   File "/opt/stack/new/nova/nova/image/glance.py", line 407, in delete
  2014-03-05 09:02:21.945 | 
  2014-03-05 09:02:21.946 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632]     raise exception.ImageNotFound(image_id=image_id)
  2014-03-05 09:02:21.946 | 
  2014-03-05 09:02:21.946 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] ImageNotFound: Image 497e7e9f-7924-4499-ad6b-cb272af2069d could not be found.
  2014-03-05 09:02:21.946 | 
  2014-03-05 09:02:21.946 | *** Not Whitelisted *** 2014-03-05 08:34:22.509 28941 TRACE nova.compute.manager [instance: fe3019e7-efda-4e00-b58b-9e9f63103632] 
  2014-03-05 09:02:21.946 | 
  2014-03-05 09:02:21.947 | *** Not Whitelisted *** 2014-03-05 08:34:22.870 28941 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Instance fe3019e7-efda-4e00-b58b-9e9f63103632 is not running.
  2014-03-05 09:02:21.947 | 
  2014-03-05 09:02:21.947 | *** Not Whitelisted *** 2014-03-05 08:34:22.872 28941 ERROR oslo.messaging._drivers.common [-] Returning exception Instance fe3019e7-efda-4e00-b58b-9e9f63103632 is not running. to caller
  2014-03-05 09:02:21.947 | 
  2014-03-05 09:02:21.947 | *** Not Whitelisted *** 2014-03-05 08:34:22.872 28941 ERROR oslo.messaging._drivers.common [-] ['Traceback (most recent call last):\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n    incoming.message))\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/opt/stack/new/nova/nova/exception.py", line 88, in wrapped\n    payload)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/exception.py", line 71, in wrapped\n    return f(self, context, *args, **kw)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 244, in decorated_function\n    pass\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 230, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 272, in decorated_function\n    e, sys.exc_info())\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 259, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 320, in decorated_function\n    % image_id, instance=instance)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 310, in decorated_function\n    *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 2481, in snapshot_instance\n    task_states.IMAGE_SNAPSHOT)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 2512, in _snapshot_instance\n    update_task_state)\n', '  File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1455, in snapshot\n    raise exception.InstanceNotRunning(instance_id=instance[\'uuid\'])\n', 'InstanceNotRunning: Instance fe3019e7-efda-4e00-b58b-9e9f63103632 is not running.\n']
  2014-03-05 09:02:21.948 | 
  2014-03-05 09:02:21.968 | 2014-03-05 08:38:23.761 28941 ERROR nova.virt.libvirt.driver [-] [instance: 5fc72019-fa1f-47fd-9441-dba16e34da89] During wait destroy, instance disappeared.
  2014-03-05 09:02:21.968 | 
  2014-03-05 09:02:21.975 | 2014-03-05 08:38:44.399 28941 ERROR nova.virt.libvirt.driver [-] [instance: abadeb0e-a22d-4b6e-b4d2-f94fb6e22b88] During wait destroy, instance disappeared.
  2014-03-05 09:02:21.975 | 
  2014-03-05 09:02:21.999 | 2014-03-05 08:40:53.999 28941 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
  2014-03-05 09:02:21.999 | 
  2014-03-05 09:02:22.087 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] Traceback (most recent call last):
  2014-03-05 09:02:22.087 | 
  2014-03-05 09:02:22.088 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]   File "/opt/stack/new/nova/nova/compute/manager.py", line 310, in decorated_function
  2014-03-05 09:02:22.088 | 
  2014-03-05 09:02:22.088 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]     *args, **kwargs)
  2014-03-05 09:02:22.088 | 
  2014-03-05 09:02:22.089 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2481, in snapshot_instance
  2014-03-05 09:02:22.089 | 
  2014-03-05 09:02:22.089 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]     task_states.IMAGE_SNAPSHOT)
  2014-03-05 09:02:22.089 | 
  2014-03-05 09:02:22.089 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2512, in _snapshot_instance
  2014-03-05 09:02:22.089 | 
  2014-03-05 09:02:22.090 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]     update_task_state)
  2014-03-05 09:02:22.090 | 
  2014-03-05 09:02:22.090 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1455, in snapshot
  2014-03-05 09:02:22.090 | 
  2014-03-05 09:02:22.090 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]     raise exception.InstanceNotRunning(instance_id=instance['uuid'])
  2014-03-05 09:02:22.090 | 
  2014-03-05 09:02:22.091 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] InstanceNotRunning: Instance 2576443e-3e73-42e5-b33f-c6f427ffaf65 is not running.
  2014-03-05 09:02:22.091 | 
  2014-03-05 09:02:22.091 | *** Not Whitelisted *** 2014-03-05 08:47:18.004 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] 
  2014-03-05 09:02:22.091 | 
  2014-03-05 09:02:22.126 | 2014-03-05 08:47:18.032 ERROR nova.compute.manager [req-54c023e1-e30d-458c-905a-8cd7df35df0a ImagesOneServerV3Test-1071173160 ImagesOneServerV3Test-341425521] [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] Error while trying to clean up image 73ebe813-c02b-4204-b8cd-651e225871e3
  2014-03-05 09:02:22.126 | 
  2014-03-05 09:02:22.126 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] Traceback (most recent call last):
  2014-03-05 09:02:22.127 | 
  2014-03-05 09:02:22.127 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]   File "/opt/stack/new/nova/nova/compute/manager.py", line 317, in decorated_function
  2014-03-05 09:02:22.127 | 
  2014-03-05 09:02:22.127 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]     image_service.delete(context, image_id)
  2014-03-05 09:02:22.127 | 
  2014-03-05 09:02:22.128 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]   File "/opt/stack/new/nova/nova/image/glance.py", line 407, in delete
  2014-03-05 09:02:22.128 | 
  2014-03-05 09:02:22.128 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65]     raise exception.ImageNotFound(image_id=image_id)
  2014-03-05 09:02:22.128 | 
  2014-03-05 09:02:22.128 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] ImageNotFound: Image 73ebe813-c02b-4204-b8cd-651e225871e3 could not be found.
  2014-03-05 09:02:22.129 | 
  2014-03-05 09:02:22.129 | *** Not Whitelisted *** 2014-03-05 08:47:18.032 28941 TRACE nova.compute.manager [instance: 2576443e-3e73-42e5-b33f-c6f427ffaf65] 
  2014-03-05 09:02:22.129 | 
  2014-03-05 09:02:22.129 | *** Not Whitelisted *** 2014-03-05 08:47:18.188 28941 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Instance 2576443e-3e73-42e5-b33f-c6f427ffaf65 is not running.
  2014-03-05 09:02:22.129 | 
  2014-03-05 09:02:22.130 | *** Not Whitelisted *** 2014-03-05 08:47:18.189 28941 ERROR oslo.messaging._drivers.common [-] Returning exception Instance 2576443e-3e73-42e5-b33f-c6f427ffaf65 is not running. to caller
  2014-03-05 09:02:22.130 | 
  2014-03-05 09:02:22.130 | *** Not Whitelisted *** 2014-03-05 08:47:18.189 28941 ERROR oslo.messaging._drivers.common [-] ['Traceback (most recent call last):\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n    incoming.message))\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/new/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/opt/stack/new/nova/nova/exception.py", line 88, in wrapped\n    payload)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/exception.py", line 71, in wrapped\n    return f(self, context, *args, **kw)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 244, in decorated_function\n    pass\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 230, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 272, in decorated_function\n    e, sys.exc_info())\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 259, in decorated_function\n    return function(self, context, *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 320, in decorated_function\n    % image_id, instance=instance)\n', '  File "/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 310, in decorated_function\n    *args, **kwargs)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 2481, in snapshot_instance\n    task_states.IMAGE_SNAPSHOT)\n', '  File "/opt/stack/new/nova/nova/compute/manager.py", line 2512, in _snapshot_instance\n    update_task_state)\n', '  File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1455, in snapshot\n    raise exception.InstanceNotRunning(instance_id=instance[\'uuid\'])\n', 'InstanceNotRunning: Instance 2576443e-3e73-42e5-b33f-c6f427ffaf65 is not running.\n']
  2014-03-05 09:02:22.130 | 
  2014-03-05 09:02:22.131 | 2014-03-05 08:50:32.939 28941 ERROR nova.virt.libvirt.driver [-] [instance: 23351a8f-3658-45cc-ac21-546faa497135] During wait destroy, instance disappeared.
  2014-03-05 09:02:22.132 | 
  2014-03-05 09:02:23.400 | Log File: ceilometer-alarm-evaluator
  2014-03-05 09:02:23.400 | 2014-03-05 08:26:16.189 30266 ERROR ceilometer.alarm.service [-] alarm evaluation cycle failed
  2014-03-05 09:02:23.400 | 
  2014-03-05 09:02:23.680 | Log File: n-api
  2014-03-05 09:02:23.680 | 2014-03-05 08:44:10.826 ERROR nova.api.openstack [req-b26e1663-8d90-4942-9909-e3491f4257eb ServersNegativeTestJSON-1261521225 ServersNegativeTestJSON-139481132] Caught error: Instance 3d9f406f-b4d0-41b1-b83a-ecf0330a99cc could not be found.
  2014-03-05 09:02:23.681 | 
  2014-03-05 09:02:23.683 | 2014-03-05 08:44:15.568 ERROR nova.api.openstack [req-4eb3cd32-f1d7-4909-95f2-4bb43e76afdf ServersNegativeTestJSON-1261521225 ServersNegativeTestJSON-139481132] Caught error: Instance 8374d83b-c35d-4670-a0b1-6bd4168ee026 could not be found.
  2014-03-05 09:02:23.683 | 
  2014-03-05 09:02:23.697 | 2014-03-05 08:45:16.125 ERROR nova.api.openstack [req-933bd333-6de4-4387-8610-3f59ce4246dc VirtualInterfacesNegativeTestJSON-1503600377 VirtualInterfacesNegativeTestJSON-1464645285] Caught error: Instance a7c4748e-34d2-4dca-9ca9-7df653ad25a2 could not be found.
  2014-03-05 09:02:23.698 | 
  2014-03-05 09:02:23.720 | 2014-03-05 08:46:22.756 ERROR nova.api.openstack [req-f38a2dfc-a841-466f-8e6e-f95e4be75736 ServersNegativeTestXML-544157769 ServersNegativeTestXML-1067209753] Caught error: Instance e17a869f-ecde-4770-9e60-89e4916a7d92 could not be found.
  2014-03-05 09:02:23.720 | 
  2014-03-05 09:02:23.723 | 2014-03-05 08:46:28.970 ERROR nova.api.openstack [req-f12c6218-b13d-42cf-9620-d31deb8f309e ServersNegativeTestXML-544157769 ServersNegativeTestXML-1067209753] Caught error: Instance f376a419-2751-49cb-9113-bb343d38af85 could not be found.
  2014-03-05 09:02:23.723 | 
  2014-03-05 09:02:23.730 | 2014-03-05 08:47:04.340 ERROR nova.api.openstack [req-9e8a00a7-45ce-41cb-a1cb-cb88a177fc7a ServersAdminV3Test-316812114 ServersAdminV3Test-1656858922] Caught error: Instance ec774d4e-8cd3-4dc2-b90f-8ad1c01548de could not be found.
  2014-03-05 09:02:23.730 | 
  2014-03-05 09:02:23.732 | 2014-03-05 08:47:10.502 ERROR nova.api.openstack [req-5e512558-f188-42fb-a739-f25a5495429c VirtualInterfacesNegativeTestXML-2023142622 VirtualInterfacesNegativeTestXML-1968584697] Caught error: Instance 6318125a-920d-4dd8-b31d-5e227c006463 could not be found.
  2014-03-05 09:02:23.732 | 
  2014-03-05 09:02:23.798 | 2014-03-05 08:52:05.139 ERROR nova.api.openstack [req-74aa8088-08c4-4a2a-adf5-c1afe2e90cfc ServersV3Test-1818645145 ServersV3Test-161538606] Caught error: Instance 333c06ea-430a-4ee1-90ca-132e61604ef3 could not be found.
  2014-03-05 09:02:23.798 | 
  2014-03-05 09:02:24.441 | Log File: g-api
  2014-03-05 09:02:24.442 | 2014-03-05 08:25:55.309 27572 ERROR swiftclient [4b4be3d1-eaf4-4f68-ab4d-c3091080cf81 1425971ac0b44225b4cf6e0649337aeb 3063ee5211114763b3ec1046563b18b5 - - -] Container HEAD failed: http://127.0.0.1:8080:8080/v1/AUTH_08965a364f3646febdcc3ea1566cb865/glance 404 Not Found
  2014-03-05 09:02:24.442 | 
  2014-03-05 09:02:24.802 | Log File: c-api
  2014-03-05 09:02:24.802 | 2014-03-05 08:54:35.679 30090 ERROR cinder.volume.api [req-1240d8f1-435f-4f74-9d91-08e9f6e8e24d None] Volume status must be available to reserve
  2014-03-05 09:02:24.802 | 
  2014-03-05 09:02:24.811 | 2014-03-05 08:54:48.380 30090 ERROR cinder.volume.api [req-4b69fd70-34fd-4954-8f24-3ab55d10cd9c None] Volume status must be available to reserve
  2014-03-05 09:02:24.811 | 
  2014-03-05 09:02:24.851 | Logs have errors
  2014-03-05 09:02:24.851 | FAILED
  2014-03-05 09:02:24.859 | ERROR: gate_hook failed

To manage notifications about this bug go to:
https://bugs.launchpad.net/openstack-ci/+bug/1288144/+subscriptions


Follow ups

References