yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #01844
[Bug 1161538] Re: migrate fails with 'ProcessExecutionError'
The log shows you're out of space on a disk that is trying to get
something copied to it.
** Changed in: nova
Status: New => Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1161538
Title:
migrate fails with 'ProcessExecutionError'
Status in OpenStack Compute (Nova):
Invalid
Bug description:
I applied the latest build to my vs347 system in an attempt to verify
a fix for Bug 1160489.
I ended up hitting a different error:
nova boot --image be8b6475-26d8-410f-aaa5-8b278d98c8f9 --flavor 1
MIGRATE1
[root@vs347 ~]# nova show MIGRATE1
+-------------------------------------+----------------------------------------------------------+
| Property | Value |
+-------------------------------------+----------------------------------------------------------+
| status | BUILD |
| updated | 2013-03-28T16:48:22Z |
| OS-EXT-STS:task_state | networking |
| OS-EXT-SRV-ATTR:host | vs342.rch.kstart.ibm.com |
| key_name | None |
| image | Rhel6MasterFile (be8b6475-26d8-410f-aaa5-8b278d98c8f9) |
| hostId | 7ddf45b44e3e8078fa9401525a630083670fdf5a5792784c506a73f7 |
| OS-EXT-STS:vm_state | building |
| OS-EXT-SRV-ATTR:instance_name | bvt-instance-0000003d |
| OS-EXT-SRV-ATTR:hypervisor_hostname | olyblade02.rch.stglabs.ibm.com |
| flavor | m1.tiny (1) |
| id | 29931d39-8d18-4a25-b733-59e894d94731 |
| security_groups | [{u'name': u'default'}] |
| user_id | 3ccf55fd609b45319f24fe681338886d |
| name | MIGRATE1 |
| created | 2013-03-28T16:48:20Z |
| tenant_id | 67b1c37f4ca64283908c7077e9e59997 |
| OS-DCF:diskConfig | MANUAL |
| metadata | {} |
| accessIPv4 | |
| accessIPv6 | |
| progress | 0 |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-AZ:availability_zone | nova |
| config_drive | |
+-------------------------------------+----------------------------------------------------------+
[root@vs347 ~]# nova list
+--------------------------------------+----------+--------+-------------------+
| ID | Name | Status | Networks |
+--------------------------------------+----------+--------+-------------------+
| 29931d39-8d18-4a25-b733-59e894d94731 | MIGRATE1 | ACTIVE | demonet=172.0.0.5 |
+--------------------------------------+----------+--------+-------------------+
[root@vs347 ~]# nova migrate MIGRATE1
[root@vs347 ~]#
* State as of 11:53 a.m.
[root@vs347 ~]# nova list
+--------------------------------------+----------+--------+-------------------+
| ID | Name | Status | Networks |
+--------------------------------------+----------+--------+-------------------+
| 29931d39-8d18-4a25-b733-59e894d94731 | MIGRATE1 | RESIZE | demonet=172.0.0.5 |
+--------------------------------------+----------+--------+-------------------+
[root@vs347 ~]# nova list
+--------------------------------------+----------+--------+-------------------+
| ID | Name | Status | Networks |
+--------------------------------------+----------+--------+-------------------+
| 29931d39-8d18-4a25-b733-59e894d94731 | MIGRATE1 | ERROR | demonet=172.0.0.5 |
+--------------------------------------+----------+--------+-------------------+
* Comes back in error state
[root@vs347 ~]# nova show MIGRATE1
+-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| status | ERROR |
| updated | 2013-03-28T17:10:54Z |
| OS-EXT-STS:task_state | None |
| OS-EXT-SRV-ATTR:host | vs342.rch.kstart.ibm.com |
| key_name | None |
| image | Rhel6MasterFile (be8b6475-26d8-410f-aaa5-8b278d98c8f9) |
| hostId | 7ddf45b44e3e8078fa9401525a630083670fdf5a5792784c506a73f7 |
| OS-EXT-STS:vm_state | error |
| OS-EXT-SRV-ATTR:instance_name | bvt-instance-0000003d |
| OS-EXT-SRV-ATTR:hypervisor_hostname | olyblade02.rch.stglabs.ibm.com |
| flavor | m1.tiny (1) |
| demonet network | 172.0.0.5 |
| id | 29931d39-8d18-4a25-b733-59e894d94731 |
| security_groups | [{u'name': u'default'}] |
| user_id | 3ccf55fd609b45319f24fe681338886d |
| name | MIGRATE1 |
| created | 2013-03-28T16:48:20Z |
| tenant_id | 67b1c37f4ca64283908c7077e9e59997 |
| OS-DCF:diskConfig | MANUAL |
| metadata | {} |
| accessIPv4 | |
| accessIPv6 | |
| fault | {u'message': u'ProcessExecutionError', u'code': 500, u'details': u'Unexpected error while running command. |
| | Command: scp -o "StrictHostKeyChecking no" -i os-9f464ddcfb2043c988fac6ca2e98950f /mnt/openstack/images/bvt-instance-0000003d_rsz.gz padmin@9.5.41.90:/mnt/openstack/images |
| | Exit code: 1 |
| | Stdout: \'\' |
| | Stderr: \'scp: /mnt/openstack/images/bvt-instance-0000003d_rsz.gz: No space left on device\ |
| | \' |
| | File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 229, in decorated_function |
| | return function(self, context, *args, **kwargs) |
| | File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 2259, in resize_instance |
| | block_device_info) |
| | File "/usr/lib/python2.6/site-packages/nova/virt/powervm/driver.py", line 256, in migrate_disk_and_power_off |
| | instance[\'name\']) |
| | File "/usr/lib/python2.6/site-packages/nova/virt/powervm/operator.py", line 604, in migrate_disk |
| | device_name, src_host, dest, image_path, instance_name) |
| | File "/usr/lib/python2.6/site-packages/nova/virt/powervm/blockdev.py", line 395, in migrate_volume |
| | self.run_vios_command(cmd) |
| | File "/usr/lib/python2.6/site-packages/nova/virt/powervm/blockdev.py", line 206, in run_vios_command |
| | check_exit_code=check_exit_code) |
| | File "/usr/lib/python2.6/site-packages/nova/utils.py", line 322, in ssh_execute |
| | cmd=cmd) |
| | ', u'created': u'2013-03-28T17:07:05Z'} |
| OS-EXT-STS:power_state | 4 |
| OS-EXT-AZ:availability_zone | nova |
| config_drive | |
+-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2013-03-28 12:07:11.987 ERROR nova.openstack.common.rpc.amqp [req-ab84709c-19a1-447b-a80a-dc2588e340ff 3ccf55fd609b45319f24fe681338886d 67b1c37f4ca64283908c7077e9e59997] Exception during message handling
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 393, in _process_data
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args)
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 133, in dispatch
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs)
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 120, in wrapped
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp temp_level, payload)
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 97, in wrapped
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp return f(self, context, *args, **kw)
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 214, in decorated_function
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp pass
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 200, in decorated_function
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 265, in decorated_function
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp function(self, context, *args, **kwargs)
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 242, in decorated_function
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp e, sys.exc_info())
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 229, in decorated_function
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 2259, in resize_instance
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp block_device_info)
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/powervm/driver.py", line 256, in migrate_disk_and_power_off
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp instance['name'])
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/powervm/operator.py", line 604, in migrate_disk
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp device_name, src_host, dest, image_path, instance_name)
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/powervm/blockdev.py", line 395, in migrate_volume
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp self.run_vios_command(cmd)
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/powervm/blockdev.py", line 206, in run_vios_command
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp check_exit_code=check_exit_code)
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/utils.py", line 322, in ssh_execute
2013-03-28 12:07:11.987 32191 TRACE nova.openstack.common.rpc.amqp cmd=cmd)
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1161538/+subscriptions
