yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #47503
[Bug 1550639] Re: After migrate volume being attached instance, the instance cann't run normally
What version of nova and cinder are you using?
I'm not familiar with the volume migration API in Cinder, but it looks
like it calls out to Nova to perform a swap volume operation.
Per your comment 1, you're expecting that Cinder will handle the failure
from the Nova swap volume operation and rollback the failure? That's
reasonable but would need the Cinder team to weigh in here.
Do you have the Cinder logs from the failure to see why those don't
rollback?
** Also affects: cinder
Importance: Undecided
Status: New
** Changed in: nova
Status: New => Incomplete
** Tags added: migration volumes
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1550639
Title:
After migrate volume being attached instance, the instance cann't
run normally
Status in Cinder:
New
Status in OpenStack Compute (nova):
Incomplete
Bug description:
Reproducing method as following:
1.create a volume from image
[root@2C5_10_DELL05 ~(keystone_admin)]# cinder create --image-id fd8330b3-a307-4140-8fe0-01341b583e26 --name test_image_volume --volume-type KSIP 1
+---------------------------------------+--------------------------------------+
| Property | Value |
+---------------------------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2016-02-27T04:20:37.000000 |
| description | None |
| encrypted | False |
| id | a0dae16a-2669-49c7-a118-250c31adc655 |
| metadata | {} |
| multiattach | False |
| name | test_image_volume |
| os-vol-host-attr:host | None |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 181a578bc97642f2b9e153bec622f130 |
| os-volume-replication:driver_data | None |
| os-volume-replication:extended_status | None |
| replication_status | disabled |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | creating |
| user_id | 8b34e1ab75024fcba0ea69a6fd0937c3 |
| volume_type | KSIP |
+---------------------------------------+--------------------------------------+
2、boot a instance from the step 1 volume.
[root@2C5_10_DELL05 ~(keystone_admin)]# nova boot --flavor 1 --block-device id=a0dae16a-2669-49c7-a118-250c31adc655,source=volume,dest=volume,bootindex=0 --nic net-id=5c8f7e7a-5a75-48eb-9c68-096278585c18 test_vm
+--------------------------------------+--------------------------------------------------+
| Property | Value |
+--------------------------------------+--------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000647 |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | JEeW4BR4WL3a |
| autostart | TRUE |
| boot_index_type | |
| config_drive | |
| created | 2016-02-27T04:22:42Z |
| flavor | m1.tiny (1) |
| hostId | |
| id | a740b3da-42e7-4cba-9408-8df3b4846dcc |
| image | Attempt to boot from volume - no image supplied |
| key_name | - |
| metadata | {} |
| move | TRUE |
| name | test_vm |
| novnc | TRUE |
| os-extended-volumes:volumes_attached | [{"id": "a0dae16a-2669-49c7-a118-250c31adc655"}] |
| priority | 50 |
| progress | 0 |
| qos | |
| security_groups | default |
| status | BUILD |
| tenant_id | 181a578bc97642f2b9e153bec622f130 |
| updated | 2016-02-27T04:22:43Z |
| user_id | 8b34e1ab75024fcba0ea69a6fd0937c3 |
+--------------------------------------+--------------------------------------------------+
3. migrate the in-use status volume
[root@2C5_10_DELL05 ~(keystone_admin)]# cinder migrate a0dae16a-2669-49c7-a118-250c31adc655 2C5_10_DELL05@KS3200ISCSIDriver-2#KS3200_IPSAN
4. migrate volume fail, nova-compute.log as following:
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher executor_callback))
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher executor_callback)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 8699, in swap_volume
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher new_volume_id)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 88, in wrapped
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher payload)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 71, in wrapped
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 379, in decorated_function
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher LOG.warning(msg, e, instance_uuid=instance_uuid)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 350, in decorated_function
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 407, in decorated_function
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 395, in decorated_function
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5965, in swap_volume
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher new_volume_id)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5932, in _swap_volume
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher self.volume_api.unreserve_volume(context, new_volume_id)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5913, in _swap_volume
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher resize_to)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1241, in swap_volume
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher self._disconnect_volume(old_connection_info, disk_dev)
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1092, in _disconnect_volume
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher raise
2016-02-27 11:32:47.986 29370 TRACE oslo_messaging.rpc.dispatcher TypeError: exceptions must be old-style classes or derived from BaseException, not NoneType
5. Then, the instance is still running ,active. but login to the virtual machine system, find the guest OS changed read-only file system.
To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1550639/+subscriptions
References
