| Thread Previous • Date Previous • Date Next • Thread Next |
Public bug reported:
Environment:
distribution:
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.2 LTS"
$ sudo virsh version
Compiled against library: libvirt 1.2.2
Using library: libvirt 1.2.2
Using API: QEMU 1.2.2
Running hypervisor: QEMU 2.0.0
$ git log --oneline
806113e Merge "Changed filter_by() to filter() during filtering instances in db API"
...
There are two hosts in my environment.
Host A is controller with compute node. Host B is only as compute
node.
Produce:
1. I upgrade the nova code in Host A. and restart n-sch, n-cond,
n-cpu.
And keep the n-cpu old version on Host B.
2. do live migration
$ nova live-migration tt
it report error.
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------+
| Property | Value
|
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL
|
| OS-EXT-AZ:availability_zone | nova
|
| OS-EXT-SRV-ATTR:host | shaohe1
|
| OS-EXT-SRV-ATTR:hostname | tt |
| OS-EXT-SRV-ATTR:hypervisor_hostname | shaohe1 |
| OS-EXT-SRV-ATTR:instance_name | instance-00000002 |
| OS-EXT-SRV-ATTR:kernel_id | 89e91cc1-a40c-4c9f-bcfa-37b0e94d5f57 |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | d4c09101-c0fa-40de-9c88-28b9428d03fb |
| OS-EXT-SRV-ATTR:reservation_id | r-p0884maw |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | migrating |
| OS-EXT-STS:vm_state | error |
| OS-SRV-USG:launched_at | 2015-12-18T07:41:00.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | True |
|
| created | 2015-12-18T07:40:44Z |
| fault | {"message": "Remote error: UnsupportedVersion Endpoint does not support RPC version 4.3. Attempted method: select_destinations |
| | [u'Traceback (most recent call last):\ |
| | ', u' File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py\", line 142, in _d", "code": 500, "details": " File \"/opt/stack/nova/nova/conductor/manager.py\", line 295, in _live_migrate |
| | task.execute() |
| | File \"/opt/stack/nova/nova/conductor/tasks/base.py\", line 27, in wrap |
| | self.rollback() |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py\", line 204, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File \"/opt/stack/nova/nova/conductor/tasks/base.py\", line 24, in wrap |
| | return original(self) |
| | File \"/opt/stack/nova/nova/conductor/tasks/base.py\", line 42, in execute |
| | return self._execute() |
| | File \"/opt/stack/nova/nova/conductor/tasks/live_migrate.py\", line 58, in _execute |
| | self.destination = self._find_destination() |
| | File \"/opt/stack/nova/nova/conductor/tasks/live_migrate.py\", line 181, in _find_destination |
| | spec_obj)[0]['host'] |
| | File \"/opt/stack/nova/nova/scheduler/utils.py\", line 358, in wrapped |
| | return func(*args, **kwargs) |
| | File \"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 51, in select_destinations |
| | return self.queryclient.select_destinations(context, spec_obj) |
| | File \"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 37, in __run_method |
| | return getattr(self.instance, __name)(*args, **kwargs)
|
| | File \"/opt/stack/nova/nova/scheduler/client/query.py\", line 32, in select_destinations |
| | return self.scheduler_rpcapi.select_destinations(context, spec_obj) |
| | File \"/opt/stack/nova/nova/scheduler/rpcapi.py\", line 121, in select_destinations |
| | return cctxt.call(ctxt, 'select_destinations', **msg_args) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py\", line 158, in call |
| | retry=self.retry) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py\", line 90, in _send |
| | timeout=timeout, retry=retry) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\", line 464, in send |
| | retry=retry) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\", line 455, in _send |
| | raise result |
| | ", "created": "2016-01-22T03:41:21Z"} |
| flavor | m1.tiny (1) |
| hostId | 94170a2af829a4a136f9f9c9d3c3f8f10df72bc1238a5eeef5b4ffa5 |
| id | 058fc419-a8a8-4e08-b62c-a9841ef9cd3f |
| image | cirros-0.3.4-x86_64-uec (3476ee05-b80e-4a80-aac4-78350ce132a2) |
| key_name | - |
| metadata | {} |
| name | tt |
| os-extended-volumes:volumes_attached | [] |
| private network | fdf1:3ce8:94a:0:f816:3eff:fee1:2a95, 10.0.0.4 |
| security_groups | default
|
| status | ERROR |
| tenant_id | f5a8829cc14c4825a2728b273aa91aa1 |
| updated | 2016-01-22T03:41:21Z |
| user_id | b5450f0c30154d2bb1506968a05c6f80 |
+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
3. when I do live migration again.
$ nova live-migration tt
ERROR (Conflict): Cannot 'os-migrateLive' instance 058fc419-a8a8-4e08-b62c-a9841ef9cd3f while it is in vm_state error (HTTP 409) (Request-ID: req-2712a326-c3ab-4a6d-9afb-5001b3822047)
4. and stop the server.
$ nova stop tt
Cannot 'stop' instance 058fc419-a8a8-4e08-b62c-a9841ef9cd3f while it is in task_state migrating (HTTP 409) (Request-ID: req-580ae847-1a5d-44c8-a285-7cc31dc9e0e2)
ERROR (CommandError): Unable to stop the specified server(s).
** Affects: nova
Importance: Undecided
Assignee: lvmxh (shaohef)
Status: New
** Changed in: nova
Assignee: (unassigned) => lvmxh (shaohef)
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1536916
Title:
nova's task_state still is migrating after live migration failed
Status in OpenStack Compute (nova):
New
Bug description:
Environment:
distribution:
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.2 LTS"
$ sudo virsh version
Compiled against library: libvirt 1.2.2
Using library: libvirt 1.2.2
Using API: QEMU 1.2.2
Running hypervisor: QEMU 2.0.0
$ git log --oneline
806113e Merge "Changed filter_by() to filter() during filtering instances in db API"
...
There are two hosts in my environment.
Host A is controller with compute node. Host B is only as compute
node.
Produce:
1. I upgrade the nova code in Host A. and restart n-sch, n-cond,
n-cpu.
And keep the n-cpu old version on Host B.
2. do live migration
$ nova live-migration tt
it report error.
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------+
| Property | Value
|
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL
|
| OS-EXT-AZ:availability_zone | nova
|
| OS-EXT-SRV-ATTR:host | shaohe1
|
| OS-EXT-SRV-ATTR:hostname | tt |
| OS-EXT-SRV-ATTR:hypervisor_hostname | shaohe1 |
| OS-EXT-SRV-ATTR:instance_name | instance-00000002 |
| OS-EXT-SRV-ATTR:kernel_id | 89e91cc1-a40c-4c9f-bcfa-37b0e94d5f57 |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | d4c09101-c0fa-40de-9c88-28b9428d03fb |
| OS-EXT-SRV-ATTR:reservation_id | r-p0884maw |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | migrating |
| OS-EXT-STS:vm_state | error |
| OS-SRV-USG:launched_at | 2015-12-18T07:41:00.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | True |
|
| created | 2015-12-18T07:40:44Z |
| fault | {"message": "Remote error: UnsupportedVersion Endpoint does not support RPC version 4.3. Attempted method: select_destinations |
| | [u'Traceback (most recent call last):\ |
| | ', u' File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py\", line 142, in _d", "code": 500, "details": " File \"/opt/stack/nova/nova/conductor/manager.py\", line 295, in _live_migrate |
| | task.execute() |
| | File \"/opt/stack/nova/nova/conductor/tasks/base.py\", line 27, in wrap |
| | self.rollback() |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py\", line 204, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File \"/opt/stack/nova/nova/conductor/tasks/base.py\", line 24, in wrap |
| | return original(self) |
| | File \"/opt/stack/nova/nova/conductor/tasks/base.py\", line 42, in execute |
| | return self._execute() |
| | File \"/opt/stack/nova/nova/conductor/tasks/live_migrate.py\", line 58, in _execute |
| | self.destination = self._find_destination() |
| | File \"/opt/stack/nova/nova/conductor/tasks/live_migrate.py\", line 181, in _find_destination |
| | spec_obj)[0]['host'] |
| | File \"/opt/stack/nova/nova/scheduler/utils.py\", line 358, in wrapped |
| | return func(*args, **kwargs) |
| | File \"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 51, in select_destinations |
| | return self.queryclient.select_destinations(context, spec_obj) |
| | File \"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 37, in __run_method |
| | return getattr(self.instance, __name)(*args, **kwargs)
|
| | File \"/opt/stack/nova/nova/scheduler/client/query.py\", line 32, in select_destinations |
| | return self.scheduler_rpcapi.select_destinations(context, spec_obj) |
| | File \"/opt/stack/nova/nova/scheduler/rpcapi.py\", line 121, in select_destinations |
| | return cctxt.call(ctxt, 'select_destinations', **msg_args) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py\", line 158, in call |
| | retry=self.retry) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py\", line 90, in _send |
| | timeout=timeout, retry=retry) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\", line 464, in send |
| | retry=retry) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\", line 455, in _send |
| | raise result |
| | ", "created": "2016-01-22T03:41:21Z"} |
| flavor | m1.tiny (1) |
| hostId | 94170a2af829a4a136f9f9c9d3c3f8f10df72bc1238a5eeef5b4ffa5 |
| id | 058fc419-a8a8-4e08-b62c-a9841ef9cd3f |
| image | cirros-0.3.4-x86_64-uec (3476ee05-b80e-4a80-aac4-78350ce132a2) |
| key_name | - |
| metadata | {} |
| name | tt |
| os-extended-volumes:volumes_attached | [] |
| private network | fdf1:3ce8:94a:0:f816:3eff:fee1:2a95, 10.0.0.4 |
| security_groups | default
|
| status | ERROR |
| tenant_id | f5a8829cc14c4825a2728b273aa91aa1 |
| updated | 2016-01-22T03:41:21Z |
| user_id | b5450f0c30154d2bb1506968a05c6f80 |
+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
3. when I do live migration again.
$ nova live-migration tt
ERROR (Conflict): Cannot 'os-migrateLive' instance 058fc419-a8a8-4e08-b62c-a9841ef9cd3f while it is in vm_state error (HTTP 409) (Request-ID: req-2712a326-c3ab-4a6d-9afb-5001b3822047)
4. and stop the server.
$ nova stop tt
Cannot 'stop' instance 058fc419-a8a8-4e08-b62c-a9841ef9cd3f while it is in task_state migrating (HTTP 409) (Request-ID: req-580ae847-1a5d-44c8-a285-7cc31dc9e0e2)
ERROR (CommandError): Unable to stop the specified server(s).
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1536916/+subscriptions
| Thread Previous • Date Previous • Date Next • Thread Next |
