openstack team mailing list archive
-
openstack team
-
Mailing list archive
-
Message #13803
LibVirt Error
Hi-
While bringing up an instance, I encountered this error at Spawning.
Please find the log details below...
Please help me resolve the issue.
2012-06-28 16:23:55 INFO nova.virt.libvirt.connection
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] Injecting key into image
83e47bfa-5eca-40cd-8bf1-785d52450d5a
2012-06-28 16:23:55 DEBUG nova.utils
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo
nova-rootwrap qemu-nbd -c /dev/nbd15
/var/lib/nova/instances/instance-0000000c/disk from (pid=27972) execute
/usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-06-28 16:23:56 DEBUG nova.utils
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo
nova-rootwrap kpartx -a /dev/nbd15 from (pid=27972) execute
/usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-06-28 16:23:56 DEBUG nova.utils
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo
nova-rootwrap mount /dev/mapper/nbd15p1 /tmp/tmpvusFCm from (pid=27972)
execute /usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-06-28 16:23:56 DEBUG nova.utils
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Result was 32 from (pid=27972) execute
/usr/lib/python2.7/dist-packages/nova/utils.py:235
2012-06-28 16:23:56 DEBUG nova.utils
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Unexpected error while running command.
Command: sudo nova-rootwrap mount /dev/mapper/nbd15p1 /tmp/tmpvusFCm
Exit code: 32
Stdout: ''
Stderr: 'mount: special device /dev/mapper/nbd15p1 does not exist\n' from
(pid=27972) trycmd /usr/lib/python2.7/dist-packages/nova/utils.py:277
2012-06-28 16:23:56 DEBUG nova.utils
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo
nova-rootwrap kpartx -d /dev/nbd15 from (pid=27972) execute
/usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-06-28 16:23:56 DEBUG nova.utils
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo
nova-rootwrap qemu-nbd -d /dev/nbd15 from (pid=27972) execute
/usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-06-28 16:23:56 DEBUG nova.virt.disk.api
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Failed to mount filesystem: Unexpected
error while running command.
Command: sudo nova-rootwrap mount /dev/mapper/nbd15p1 /tmp/tmpvusFCm
Exit code: 32
Stdout: ''
Stderr: 'mount: special device /dev/mapper/nbd15p1 does not exist\n' from
(pid=27972) mount /usr/lib/python2.7/dist-packages/nova/virt/disk/api.py:205
2012-06-28 16:23:56 DEBUG nova.utils
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo
nova-rootwrap guestmount --rw -a
/var/lib/nova/instances/instance-0000000c/disk -m /dev/sda1 /tmp/tmpvusFCm
from (pid=27972) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-06-28 16:23:56 DEBUG nova.utils
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Result was 1 from (pid=27972) execute
/usr/lib/python2.7/dist-packages/nova/utils.py:235
2012-06-28 16:23:56 DEBUG nova.utils
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Unexpected error while running command.
Command: sudo nova-rootwrap guestmount --rw -a
/var/lib/nova/instances/instance-0000000c/disk -m /dev/sda1 /tmp/tmpvusFCm
Exit code: 1
Stdout: ''
Stderr: 'Traceback (most recent call last):\n File
"/usr/bin/nova-rootwrap", line 69, in <module>\n
env=filtermatch.get_environment(userargs))\n File
"/usr/lib/python2.7/subprocess.py", line 679, in __init__\n errread,
errwrite)\n File "/usr/lib/python2.7/subprocess.py", line 1249, in
_execute_child\n raise child_exception\nOSError: [Errno 2] No such file
or directory\n' from (pid=27972) trycmd
/usr/lib/python2.7/dist-packages/nova/utils.py:277
2012-06-28 16:23:56 DEBUG nova.utils
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo
nova-rootwrap fusermount -u /tmp/tmpvusFCm from (pid=27972) execute
/usr/lib/python2.7/dist-packages/nova/utils.py:219
2012-06-28 16:23:56 DEBUG nova.utils
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Result was 1 from (pid=27972) execute
/usr/lib/python2.7/dist-packages/nova/utils.py:235
2012-06-28 16:23:56 DEBUG nova.utils
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Unexpected error while running command.
Command: sudo nova-rootwrap fusermount -u /tmp/tmpvusFCm
Exit code: 1
Stdout: ''
Stderr: '/bin/fusermount: failed to unmount /tmp/tmpvusFCm: Invalid
argument\n' from (pid=27972) trycmd
/usr/lib/python2.7/dist-packages/nova/utils.py:277
2012-06-28 16:23:56 DEBUG nova.virt.disk.api
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Failed to mount filesystem: Unexpected
error while running command.
Command: sudo nova-rootwrap guestmount --rw -a
/var/lib/nova/instances/instance-0000000c/disk -m /dev/sda1 /tmp/tmpvusFCm
Exit code: 1
Stdout: ''
Stderr: 'Traceback (most recent call last):\n File
"/usr/bin/nova-rootwrap", line 69, in <module>\n
env=filtermatch.get_environment(userargs))\n File
"/usr/lib/python2.7/subprocess.py", line 679, in __init__\n errread,
errwrite)\n File "/usr/lib/python2.7/subprocess.py", line 1249, in
_execute_child\n raise child_exception\nOSError: [Errno 2] No such file
or directory\n' from (pid=27972) mount
/usr/lib/python2.7/dist-packages/nova/virt/disk/api.py:205
2012-06-28 16:23:56 WARNING nova.virt.libvirt.connection
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] Ignoring error injecting data into
image 83e47bfa-5eca-40cd-8bf1-785d52450d5a (
--
Failed to mount filesystem: Unexpected error while running command.
Command: sudo nova-rootwrap mount /dev/mapper/nbd15p1 /tmp/tmpvusFCm
Exit code: 32
Stdout: ''
Stderr: 'mount: special device /dev/mapper/nbd15p1 does not exist\n'
--
Failed to mount filesystem: Unexpected error while running command.
Command: sudo nova-rootwrap guestmount --rw -a
/var/lib/nova/instances/instance-0000000c/disk -m /dev/sda1 /tmp/tmpvusFCm
Exit code: 1
Stdout: ''
Stderr: 'Traceback (most recent call last):\n File
"/usr/bin/nova-rootwrap", line 69, in <module>\n
env=filtermatch.get_environment(userargs))\n File
"/usr/lib/python2.7/subprocess.py", line 679, in __init__\n errread,
errwrite)\n File "/usr/lib/python2.7/subprocess.py", line 1249, in
_execute_child\n raise child_exception\nOSError: [Errno 2] No such file
or directory\n')
2012-06-28 16:24:00 ERROR nova.compute.manager
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] Instance failed to spawn
2012-06-28 16:24:00 TRACE nova.compute.manager [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] Traceback (most recent call last):
2012-06-28 16:24:00 TRACE nova.compute.manager [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 592, in
_spawn
2012-06-28 16:24:00 TRACE nova.compute.manager [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e]
self._legacy_nw_info(network_info), block_device_info)
2012-06-28 16:24:00 TRACE nova.compute.manager [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped
2012-06-28 16:24:00 TRACE nova.compute.manager [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] return f(*args, **kw)
2012-06-28 16:24:00 TRACE nova.compute.manager [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line
922, in spawn
2012-06-28 16:24:00 TRACE nova.compute.manager [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] self._create_new_domain(xml)
2012-06-28 16:24:00 TRACE nova.compute.manager [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line
1575, in _create_new_domain
2012-06-28 16:24:00 TRACE nova.compute.manager [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e]
domain.createWithFlags(launch_flags)
2012-06-28 16:24:00 TRACE nova.compute.manager [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] File
"/usr/lib/python2.7/dist-packages/libvirt.py", line 581, in createWithFlags
2012-06-28 16:24:00 TRACE nova.compute.manager [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] if ret == -1: raise libvirtError
('virDomainCreateWithFlags() failed', dom=self)
2012-06-28 16:24:00 TRACE nova.compute.manager [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] libvirtError: Unable to read from
monitor: Connection reset by peer
2012-06-28 16:24:00 TRACE nova.compute.manager [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e]
2012-06-28 16:24:00 DEBUG nova.compute.manager
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] [instance:
7741f67f-ad78-4777-a5a0-6636eb8b460e] Deallocating network for instance
from (pid=27972) _deallocate_network
/usr/lib/python2.7/dist-packages/nova/compute/manager.py:616
2012-06-28 16:24:00 DEBUG nova.rpc.amqp
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Making asynchronous cast on network...
from (pid=27972) cast /usr/lib/python2.7/dist-packages/nova/rpc/amqp.py:346
2012-06-28 16:24:01 ERROR nova.rpc.amqp
[req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8
8f2e0a43c2bc43a8b75b2c08d2598ca4] Exception during message handling
2012-06-28 16:24:01 TRACE nova.rpc.amqp Traceback (most recent call last):
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 252, in
_process_data
2012-06-28 16:24:01 TRACE nova.rpc.amqp rval = node_func(context=ctxt,
**node_args)
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped
2012-06-28 16:24:01 TRACE nova.rpc.amqp return f(*args, **kw)
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 177, in
decorated_function
2012-06-28 16:24:01 TRACE nova.rpc.amqp sys.exc_info())
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-06-28 16:24:01 TRACE nova.rpc.amqp self.gen.next()
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 171, in
decorated_function
2012-06-28 16:24:01 TRACE nova.rpc.amqp return function(self, context,
instance_uuid, *args, **kwargs)
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 651, in
run_instance
2012-06-28 16:24:01 TRACE nova.rpc.amqp do_run_instance()
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/utils.py", line 945, in inner
2012-06-28 16:24:01 TRACE nova.rpc.amqp retval = f(*args, **kwargs)
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 650, in
do_run_instance
2012-06-28 16:24:01 TRACE nova.rpc.amqp self._run_instance(context,
instance_uuid, **kwargs)
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 451, in
_run_instance
2012-06-28 16:24:01 TRACE nova.rpc.amqp
self._set_instance_error_state(context, instance_uuid)
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-06-28 16:24:01 TRACE nova.rpc.amqp self.gen.next()
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 432, in
_run_instance
2012-06-28 16:24:01 TRACE nova.rpc.amqp
self._deallocate_network(context, instance)
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-06-28 16:24:01 TRACE nova.rpc.amqp self.gen.next()
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 429, in
_run_instance
2012-06-28 16:24:01 TRACE nova.rpc.amqp injected_files, admin_password)
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 592, in
_spawn
2012-06-28 16:24:01 TRACE nova.rpc.amqp
self._legacy_nw_info(network_info), block_device_info)
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped
2012-06-28 16:24:01 TRACE nova.rpc.amqp return f(*args, **kw)
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line
922, in spawn
2012-06-28 16:24:01 TRACE nova.rpc.amqp self._create_new_domain(xml)
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line
1575, in _create_new_domain
2012-06-28 16:24:01 TRACE nova.rpc.amqp
domain.createWithFlags(launch_flags)
2012-06-28 16:24:01 TRACE nova.rpc.amqp File
"/usr/lib/python2.7/dist-packages/libvirt.py", line 581, in createWithFlags
2012-06-28 16:24:01 TRACE nova.rpc.amqp if ret == -1: raise
libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2012-06-28 16:24:01 TRACE nova.rpc.amqp libvirtError: Unable to read from
monitor: Connection reset by peer
2012-06-28 16:24:01 TRACE nova.rpc.amqp
2012-06-28 16:24:27 DEBUG nova.manager [-] Running periodic task
ComputeManager._publish_service_capabilities from (pid=27972)
periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:152
2012-06-28 16:24:27 DEBUG nova.manager [-] Notifying Schedulers of
capabilities ... from (pid=27972) _publish_service_capabilities
/usr/lib/python2.7/dist-packages/nova/manager.py:203
2012-06-28 16:24:27 DEBUG nova.rpc.amqp [-] Making asynchronous fanout
cast... from (pid=27972) fanout_cast
/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py:354
2012-06-28 16:24:27 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_rescued_instances from (pid=27972) periodic_tasks
/usr/lib/python2.7/dist-packages/nova/manager.py:152
2012-06-28 16:24:27 DEBUG nova.manager [-] Skipping
ComputeManager._sync_power_states, 7 ticks left until next run from
(pid=27972) periodic_tasks
/usr/lib/python2.7/dist-packages/nova/manager.py:147
2012-06-28 16:24:27 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_bandwidth_usage from (pid=27972) periodic_tasks
/usr/lib/python2.7/dist-packages/nova/manager.py:152
2012-06-28 16:24:27 DEBUG nova.manager [-] Running periodic task
ComputeManager.update_available_resource from (pid=27972) periodic_tasks
/usr/lib/python2.7/dist-packages/nova/manager.py:152
2012-06-28 16:24:29 INFO nova.virt.libvirt.connection [-] Compute_service
record updated for openstackHost1
2012-06-28 16:24:29 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_rebooting_instances from (pid=27972) periodic_tasks
/usr/lib/python2.7/dist-packages/nova/manager.py:152
2012-06-28 16:24:29 DEBUG nova.manager [-] Skipping
ComputeManager._cleanup_running_deleted_instances, 27 ticks left until next
run from (pid=27972) periodic_tasks
/usr/lib/python2.7/dist-packages/nova/manager.py:147
2012-06-28 16:24:29 DEBUG nova.manager [-] Running periodic task
ComputeManager._heal_instance_info_cache from (pid=27972) periodic_tasks
/usr/lib/python2.7/dist-packages/nova/manager.py:152
2012-06-28 16:24:29 DEBUG nova.rpc.amqp [-] Making asynchronous call on
network ... from (pid=27972) multicall
/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py:321
2012-06-28 16:24:29 DEBUG nova.rpc.amqp [-] MSG_ID is
8afecb4fea3b4df7938556b4da364590 from (pid=27972) multicall
/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py:324
^C
root@openstackHost1:~# 2012-06-28 16:24:29 DEBUG nova.compute.manager [-]
Updated the info_cache for instance 7741f67f-ad78-4777-a5a0-6636eb8b460e
from (pid=27972) _heal_instance_info_cache
/usr/lib/python2.7/dist-packages/nova/compute/manager.py:2227
2012-06-28 16:24:29 DEBUG nova.manager [-] Skipping
ComputeManager._run_image_cache_manager_pass, 37 ticks left until next run
from (pid=27972) periodic_tasks
/usr/lib/python2.7/dist-packages/nova/manager.py:147
2012-06-28 16:24:29 DEBUG nova.manager [-] Running periodic task
ComputeManager._reclaim_queued_deletes from (pid=27972) periodic_tasks
/usr/lib/python2.7/dist-packages/nova/manager.py:152
2012-06-28 16:24:29 DEBUG nova.compute.manager [-]
FLAGS.reclaim_instance_interval <= 0, skipping... from (pid=27972)
_reclaim_queued_deletes
/usr/lib/python2.7/dist-packages/nova/compute/manager.py:2380
2012-06-28 16:24:29 DEBUG nova.manager [-] Running periodic task
ComputeManager._report_driver_status from (pid=27972) periodic_tasks
/usr/lib/python2.7/dist-packages/nova/manager.py:152
2012-06-28 16:24:29 DEBUG nova.manager [-] Running periodic task
ComputeManager._poll_unconfirmed_resizes from (pid=27972) periodic_tasks
/usr/lib/python2.7/dist-packages/nova/manager.py:152
Thanking you
--
Regards,
----------------------------------------------
Trinath Somanchi,
+91 9866 235 130
Follow ups
