yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #38890
[Bug 1397893] Re: Undeletable volume-backed instance
This was fixed in juno here: https://review.openstack.org/#/c/104048/
** Changed in: nova
Status: Incomplete => Fix Committed
** Changed in: nova
Assignee: (unassigned) => Roman Podoliaka (rpodolyaka)
** Changed in: nova
Assignee: Roman Podoliaka (rpodolyaka) => Vladik Romanovsky (vladik-romanovsky)
** Tags added: in-stable-juno
** Changed in: nova
Status: Fix Committed => Fix Released
** Changed in: nova
Importance: Undecided => High
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1397893
Title:
Undeletable volume-backed instance
Status in OpenStack Compute (nova):
Fix Released
Status in nova package in Ubuntu:
In Progress
Bug description:
nova-compute package version 1:2014.1.3-0ubuntu1.1 on Ubuntu 14.04.
Trying to delete a volume-backed instance in error state doesn't work.
Nova-compute logs the following error.
# nova delete 71a41e09-e8bc-4829-979e-1d175246da00
# tail /var/log/nova/nova-compute.log
[...]
2014-12-01 09:52:25.563 25832 AUDIT nova.compute.manager [req-39acd9da-518c-4804-bf30-1a38eace21bf 4474a81aca524682875658eb8064c33d 7dbed2bcbd7541289c34ae8392acf612] [instance: 71a41e09-e8bc-4829-979e-1d175246da00] Terminating
instance
2014-12-01 09:52:25.569 25832 ERROR nova.virt.libvirt.driver [-] [instance: 71a41e09-e8bc-4829-979e-1d175246da00] During wait destroy, instance disappeared.
2014-12-01 09:52:25.638 25832 ERROR nova.compute.manager [req-39acd9da-518c-4804-bf30-1a38eace21bf 4474a81aca524682875658eb8064c33d 7dbed2bcbd7541289c34ae8392acf612] [instance: 71a41e09-e8bc-4829-979e-1d175246da00] Setting inst
ance vm_state to ERROR
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] Traceback (most recent call last):
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2273, in do_terminate_instance
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] self._delete_instance(context, instance, bdms, quotas)
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] File "/usr/lib/python2.7/dist-packages/nova/hooks.py", line 103, in inner
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] rv = f(*args, **kwargs)
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2242, in _delete_instance
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] quotas.rollback()
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] six.reraise(self.type_, self.value, self.tb)
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2214, in _delete_instance
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] self._shutdown_instance(context, db_inst, bdms)
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2156, in _shutdown_instance
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] requested_networks)
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] six.reraise(self.type_, self.value, self.tb)
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2146, in _shutdown_instance
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] block_device_info)
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 963, in destroy
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] destroy_disks)
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1040, in cleanup
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] disk_dev = vol['mount_device'].rpartition("/")[2]
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00] AttributeError: 'NoneType' object has no attribute 'rpartition'
2014-12-01 09:52:25.638 25832 TRACE nova.compute.manager [instance: 71a41e09-e8bc-4829-979e-1d175246da00]
[...]
# nova show 71a41e09-e8bc-4829-979e-1d175246da00
+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | openstack110 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | openstack110.local |
| OS-EXT-SRV-ATTR:instance_name | instance-00000b40 |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | error |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2014-10-13T14:14:54Z |
| fault | {"message": "'NoneType' object has no attribute 'rpartition'", "code": 500, "details": " File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 290, in decorated_function |
| | return function(self, context, *args, **kwargs) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2285, in terminate_instance |
| | do_terminate_instance(instance, bdms) |
| | File \"/usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py\", line 249, in inner |
| | return f(*args, **kwargs) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2283, in do_terminate_instance |
| | self._set_instance_error_state(context, instance['uuid']) |
| | File \"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py\", line 68, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2273, in do_terminate_instance |
| | self._delete_instance(context, instance, bdms, quotas) |
| | File \"/usr/lib/python2.7/dist-packages/nova/hooks.py\", line 103, in inner |
| | rv = f(*args, **kwargs) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2242, in _delete_instance |
| | quotas.rollback() |
| | File \"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py\", line 68, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2214, in _delete_instance |
| | self._shutdown_instance(context, db_inst, bdms) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2156, in _shutdown_instance |
| | requested_networks) |
| | File \"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py\", line 68, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2146, in _shutdown_instance |
| | block_device_info) |
| | File \"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py\", line 963, in destroy |
| | destroy_disks) |
| | File \"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py\", line 1040, in cleanup |
| | disk_dev = vol['mount_device'].rpartition(\"/\")[2] |
| | ", "created": "2014-12-01T08:52:25Z"} |
| flavor | m1.large (4) |
| hostId | 567b019ea0119d6b7c09dcf0d8d859ad64a8cca16254148f439cd8b6 |
| id | 71a41e09-e8bc-4829-979e-1d175246da00 |
| image | openstack100-clone-1 (b472f236-ec54-41a3-9e92-40f091d57d05) |
| key_name | francois |
| metadata | {} |
| name | openstack100-clone |
| os-extended-volumes:volumes_attached | [{"id": "d8d89370-f334-4f78-bb35-e4a74f60d077"}] |
| status | ERROR |
| tenant_id | 7dbed2bcbd7541289c34ae8392acf612 |
| updated | 2014-12-01T08:52:25Z |
| user_id | 4474a81aca524682875658eb8064c33d |
+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
# cinder show d8d89370-f334-4f78-bb35-e4a74f60d077
+--------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{u'id': u'd8d89370-f334-4f78-bb35-e4a74f60d077', u'server_id': u'71a41e09-e8bc-4829-979e-1d175246da00', u'host_name': None, u'volume_id': u'd8d89370-f334-4f78-bb35-e4a74f60d077'}] |
| availability_zone | nova |
| bootable | true |
| created_at | 2014-10-13T14:14:55.000000 |
| display_description | |
| display_name | |
| encrypted | False |
| id | d8d89370-f334-4f78-bb35-e4a74f60d077 |
| metadata | {u'readonly': u'False', u'attached_mode': u'rw'} |
| os-vol-host-attr:host | openstack100 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 7dbed2bcbd7541289c34ae8392acf612 |
| size | 100 |
| snapshot_id | None |
| source_volid | None |
| status | available |
| volume_image_metadata | {u'container_format': u'bare', u'min_ram': u'0', u'disk_format': u'qcow2', u'image_name': u'openstack100-clone-1', u'image_id': u'b472f236-ec54-41a3-9e92-40f091d57d05', u'checksum': u'e3898f7109a6b496b39245988bbdd1e0', u'min_disk': u'80', u'size': u'68893671424'} |
| volume_type | None |
+--------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1397893/+subscriptions