yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1578691] [NEW] live-migration failed Unexpected API Error

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Eric <E3GH75@xxxxxxxxx>
Date: Thu, 05 May 2016 15:25:49 -0000
Reply-to: Bug 1578691 <1578691@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

nova live-migration fails with the following error:

[root@ler-cloudnet-01 ~]# nova live-migration badacdde-0b93-4a96-a790-3c1f6a71ea47 ler-cloudnet-01
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-186d7d86-ad97-4608-a71d-2298c6742b0f)
[root@ler-cloudnet-01 ~]# nova --debug live-migration badacdde-0b93-4a96-a790-3c1f6a71ea47 ler-cloudnet-01
DEBUG (extension:157) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token')
DEBUG (extension:157) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken')
DEBUG (extension:157) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode')
DEBUG (extension:157) found extension EntryPoint.parse('v2password = keystoneauth1.loading._plugins.identity.v2:Password')
DEBUG (extension:157) found extension EntryPoint.parse('v3password = keystoneauth1.loading._plugins.identity.v3:Password')
DEBUG (extension:157) found extension EntryPoint.parse('v3oidcpassword = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectPassword')
DEBUG (extension:157) found extension EntryPoint.parse('token = keystoneauth1.loading._plugins.identity.generic:Token')
DEBUG (extension:157) found extension EntryPoint.parse('v3token = keystoneauth1.loading._plugins.identity.v3:Token')
DEBUG (extension:157) found extension EntryPoint.parse('password = keystoneauth1.loading._plugins.identity.generic:Password')
DEBUG (session:248) REQ: curl -g -i -X GET http://ler-cloudnet-01:35357/v3 -H "Accept: application/json" -H "User-Agent: keystoneauth1/2.3.0 python-requests/2.9.1 CPython/2.7.5"
INFO (connectionpool:213) Starting new HTTP connection (1): ler-cloudnet-01
DEBUG (connectionpool:393) "GET /v3 HTTP/1.1" 200 255
DEBUG (session:277) RESP: [200] Content-Length: 255 Vary: X-Auth-Token Keep-Alive: timeout=5, max=100 Server: Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5 Connection: Keep-Alive Date: Thu, 05 May 2016 15:17:34 GMT Content-Type: application/json x-openstack-request-id: req-7bf67088-e80e-403c-912f-6763a44e2493 
RESP BODY: {"version": {"status": "stable", "updated": "2016-04-04T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.6", "links": [{"href": "http://ler-cloudnet-01:35357/v3/";, "rel": "self"}]}}

DEBUG (base:165) Making authentication request to http://ler-cloudnet-01:35357/v3/auth/tokens
DEBUG (connectionpool:393) "POST /v3/auth/tokens HTTP/1.1" 201 4234
DEBUG (session:248) REQ: curl -g -i -X GET http://ler-cloudnet-01:8774/v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419 -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a45b40d5aa2cbf5b76e7e4a0809f5eb4cd5001cb"
INFO (connectionpool:213) Starting new HTTP connection (1): ler-cloudnet-01
DEBUG (connectionpool:393) "GET /v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419 HTTP/1.1" 404 52
DEBUG (session:277) RESP: [404] Date: Thu, 05 May 2016 15:17:35 GMT Connection: keep-alive Content-Type: text/plain; charset=UTF-8 Content-Length: 52 X-Compute-Request-Id: req-578989bf-2fbd-429c-af57-b9c16744dd8f 
RESP BODY: 404 Not Found

The resource could not be found.


DEBUG (session:248) REQ: curl -g -i -X GET http://ler-cloudnet-01:8774/v2.1/ -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a45b40d5aa2cbf5b76e7e4a0809f5eb4cd5001cb"
DEBUG (connectionpool:393) "GET /v2.1/ HTTP/1.1" 200 389
DEBUG (session:277) RESP: [200] Content-Length: 389 X-Compute-Request-Id: req-d2c6d076-2edc-41f2-967d-0b7db72efb17 Vary: X-OpenStack-Nova-API-Version Connection: keep-alive X-Openstack-Nova-Api-Version: 2.1 Date: Thu, 05 May 2016 15:17:35 GMT Content-Type: application/json 
RESP BODY: {"version": {"status": "CURRENT", "updated": "2013-07-23T11:33:21Z", "links": [{"href": "http://ler-cloudnet-01:8774/v2.1/";, "rel": "self"}, {"href": "http://docs.openstack.org/";, "type": "text/html", "rel": "describedby"}], "min_version": "2.1", "version": "2.25", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2.1"}], "id": "v2.1"}}

DEBUG (extension:157) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token')
DEBUG (extension:157) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken')
DEBUG (extension:157) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode')
DEBUG (extension:157) found extension EntryPoint.parse('v2password = keystoneauth1.loading._plugins.identity.v2:Password')
DEBUG (extension:157) found extension EntryPoint.parse('v3password = keystoneauth1.loading._plugins.identity.v3:Password')
DEBUG (extension:157) found extension EntryPoint.parse('v3oidcpassword = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectPassword')
DEBUG (extension:157) found extension EntryPoint.parse('token = keystoneauth1.loading._plugins.identity.generic:Token')
DEBUG (extension:157) found extension EntryPoint.parse('v3token = keystoneauth1.loading._plugins.identity.v3:Token')
DEBUG (extension:157) found extension EntryPoint.parse('password = keystoneauth1.loading._plugins.identity.generic:Password')
DEBUG (session:248) REQ: curl -g -i -X GET http://ler-cloudnet-01:8774/v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419/servers/badacdde-0b93-4a96-a790-3c1f6a71ea47 -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.25" -H "X-Auth-Token: {SHA1}a45b40d5aa2cbf5b76e7e4a0809f5eb4cd5001cb"
DEBUG (connectionpool:393) "GET /v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419/servers/badacdde-0b93-4a96-a790-3c1f6a71ea47 HTTP/1.1" 200 2308
DEBUG (session:277) RESP: [200] Content-Length: 2308 X-Compute-Request-Id: req-6efd0796-b4f4-4fcc-883b-012af5545f3e Vary: X-OpenStack-Nova-API-Version Connection: keep-alive X-Openstack-Nova-Api-Version: 2.25 Date: Thu, 05 May 2016 15:17:35 GMT Content-Type: application/json 
RESP BODY: {"server": {"status": "ACTIVE", "OS-EXT-SRV-ATTR:ramdisk_id": "", "updated": "2016-05-05T15:15:43Z", "hostId": "863ae4569a222fdb36b65427b93b537ad801af26001dbd1543f3683a", "OS-EXT-SRV-ATTR:host": "ler-cloudsrv-01", "addresses": {"demo-net": [{"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:19:21:ca", "version": 4, "addr": "192.168.254.11", "OS-EXT-IPS:type": "fixed"}, {"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:19:21:ca", "version": 4, "addr": "172.16.7.103", "OS-EXT-IPS:type": "floating"}]}, "links": [{"href": "http://ler-cloudnet-01:8774/v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419/servers/badacdde-0b93-4a96-a790-3c1f6a71ea47";, "rel": "self"}, {"href": "http://ler-cloudnet-01:8774/32d0fe3c882b4e98b4ad9e3e3dd6e419/servers/badacdde-0b93-4a96-a790-3c1f6a71ea47";, "rel": "bookmark"}], "key_name": "centos", "image": {"id": "ecd8aacb-4089-49f0-bf0e-23572df97b42", "links": [{"href": "http://ler-cloudnet-01:8774/32d0fe3c882b4e98b4ad9e3e3dd6e419/images/ecd8aacb-4089-49f0-bf0e-23572df97b42";, "rel": "bookmark"}]}, "OS-EXT-SRV-ATTR:user_data": null, "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "active", "OS-EXT-SRV-ATTR:instance_name": "instance-00000032", "OS-EXT-SRV-ATTR:root_device_name": "/dev/vda", "OS-SRV-USG:launched_at": "2016-05-02T15:37:30.000000", "locked": false, "flavor": {"id": "d5a0079b-b66b-456d-9077-b75e4116ec04", "links": [{"href": "http://ler-cloudnet-01:8774/32d0fe3c882b4e98b4ad9e3e3dd6e419/flavors/d5a0079b-b66b-456d-9077-b75e4116ec04";, "rel": "bookmark"}]}, "id": "badacdde-0b93-4a96-a790-3c1f6a71ea47", "security_groups": [{"name": "default"}], "description": "vm001", "OS-EXT-SRV-ATTR:kernel_id": "", "host_status": "UP", "OS-EXT-AZ:availability_zone": "nova", "user_id": "b4ecd5437a964c9aa463e630c15b1c5a", "name": "vm001", "OS-EXT-SRV-ATTR:launch_index": 0, "created": "2016-05-02T15:37:18Z", "tenant_id": "1d79170d4847430f8551c28bbd1d4f4d", "OS-DCF:diskConfig": "AUTO", "OS-EXT-SRV-ATTR:hypervisor_hostname": "ler-cloudsrv-01", "os-extended-volumes:volumes_attached": [{"id": "79a4dafd-5f4d-4603-8362-48b63129d000", "delete_on_termination": false}], "accessIPv4": "", "accessIPv6": "", "OS-EXT-SRV-ATTR:reservation_id": "r-40w0zyaa", "OS-EXT-SRV-ATTR:hostname": "vm001", "progress": 0, "OS-EXT-STS:power_state": 1, "config_drive": "", "OS-SRV-USG:terminated_at": null, "metadata": {}}}

DEBUG (session:248) REQ: curl -g -i -X POST http://ler-cloudnet-01:8774/v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419/servers/badacdde-0b93-4a96-a790-3c1f6a71ea47/action -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.25" -H "X-Auth-Token: {SHA1}a45b40d5aa2cbf5b76e7e4a0809f5eb4cd5001cb" -d '{"os-migrateLive": {"block_migration": "auto", "host": "ler-cloudnet-01"}}'
DEBUG (connectionpool:393) "POST /v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419/servers/badacdde-0b93-4a96-a790-3c1f6a71ea47/action HTTP/1.1" 500 216
DEBUG (session:277) RESP: [500] Content-Length: 216 X-Compute-Request-Id: req-70b112fa-f765-41af-80ba-9e8221dff675 Vary: X-OpenStack-Nova-API-Version Connection: keep-alive X-Openstack-Nova-Api-Version: 2.25 Date: Thu, 05 May 2016 15:18:36 GMT Content-Type: application/json; charset=UTF-8 
RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'oslo_messaging.exceptions.MessagingTimeout'>", "code": 500}}

DEBUG (shell:1082) Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-70b112fa-f765-41af-80ba-9e8221dff675)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 1080, in main
    OpenStackComputeShell().main(argv)
  File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 1007, in main
    args.func(self.cs, args)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/shell.py", line 3853, in do_live_migration
    args.block_migrate)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 442, in live_migrate
    block_migration)
  File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 370, in substitution
    return methods[-1].func(obj, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1530, in live_migrate
    'block_migration': block_migration})
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1682, in _action
    info=info, **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1693, in _action_return_resp_and_body
    return self.api.client.post(url, body=body)
  File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 179, in post
    return self.request(url, 'POST', **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 94, in request
    raise exceptions.from_response(resp, body, url, method)
ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-70b112fa-f765-41af-80ba-9e8221dff675)
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-70b112fa-f765-41af-80ba-9e8221dff675)


[root@ler-cloudnet-01 ~]# yum list installed | grep nova
openstack-nova-api.noarch       1:13.0.0-1.el7          @centos-openstack-mitaka
openstack-nova-common.noarch    1:13.0.0-1.el7          @centos-openstack-mitaka
openstack-nova-compute.noarch   1:13.0.0-1.el7          @centos-openstack-mitaka
openstack-nova-conductor.noarch 1:13.0.0-1.el7          @centos-openstack-mitaka
openstack-nova-console.noarch   1:13.0.0-1.el7          @centos-openstack-mitaka
openstack-nova-novncproxy.noarch
openstack-nova-scheduler.noarch 1:13.0.0-1.el7          @centos-openstack-mitaka
python-nova.noarch              1:13.0.0-1.el7          @centos-openstack-mitaka
python-novaclient.noarch        1:3.3.0-1.el7           @centos-openstack-mitaka


Using Ceph-hammer, kvm, neutron ml2 linux-bridge

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1578691

Title:
  live-migration failed Unexpected API Error

Status in OpenStack Compute (nova):
  New

Bug description:
  nova live-migration fails with the following error:

  [root@ler-cloudnet-01 ~]# nova live-migration badacdde-0b93-4a96-a790-3c1f6a71ea47 ler-cloudnet-01
  ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-186d7d86-ad97-4608-a71d-2298c6742b0f)
  [root@ler-cloudnet-01 ~]# nova --debug live-migration badacdde-0b93-4a96-a790-3c1f6a71ea47 ler-cloudnet-01
  DEBUG (extension:157) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token')
  DEBUG (extension:157) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken')
  DEBUG (extension:157) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode')
  DEBUG (extension:157) found extension EntryPoint.parse('v2password = keystoneauth1.loading._plugins.identity.v2:Password')
  DEBUG (extension:157) found extension EntryPoint.parse('v3password = keystoneauth1.loading._plugins.identity.v3:Password')
  DEBUG (extension:157) found extension EntryPoint.parse('v3oidcpassword = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectPassword')
  DEBUG (extension:157) found extension EntryPoint.parse('token = keystoneauth1.loading._plugins.identity.generic:Token')
  DEBUG (extension:157) found extension EntryPoint.parse('v3token = keystoneauth1.loading._plugins.identity.v3:Token')
  DEBUG (extension:157) found extension EntryPoint.parse('password = keystoneauth1.loading._plugins.identity.generic:Password')
  DEBUG (session:248) REQ: curl -g -i -X GET http://ler-cloudnet-01:35357/v3 -H "Accept: application/json" -H "User-Agent: keystoneauth1/2.3.0 python-requests/2.9.1 CPython/2.7.5"
  INFO (connectionpool:213) Starting new HTTP connection (1): ler-cloudnet-01
  DEBUG (connectionpool:393) "GET /v3 HTTP/1.1" 200 255
  DEBUG (session:277) RESP: [200] Content-Length: 255 Vary: X-Auth-Token Keep-Alive: timeout=5, max=100 Server: Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5 Connection: Keep-Alive Date: Thu, 05 May 2016 15:17:34 GMT Content-Type: application/json x-openstack-request-id: req-7bf67088-e80e-403c-912f-6763a44e2493 
  RESP BODY: {"version": {"status": "stable", "updated": "2016-04-04T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.6", "links": [{"href": "http://ler-cloudnet-01:35357/v3/";, "rel": "self"}]}}

  DEBUG (base:165) Making authentication request to http://ler-cloudnet-01:35357/v3/auth/tokens
  DEBUG (connectionpool:393) "POST /v3/auth/tokens HTTP/1.1" 201 4234
  DEBUG (session:248) REQ: curl -g -i -X GET http://ler-cloudnet-01:8774/v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419 -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a45b40d5aa2cbf5b76e7e4a0809f5eb4cd5001cb"
  INFO (connectionpool:213) Starting new HTTP connection (1): ler-cloudnet-01
  DEBUG (connectionpool:393) "GET /v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419 HTTP/1.1" 404 52
  DEBUG (session:277) RESP: [404] Date: Thu, 05 May 2016 15:17:35 GMT Connection: keep-alive Content-Type: text/plain; charset=UTF-8 Content-Length: 52 X-Compute-Request-Id: req-578989bf-2fbd-429c-af57-b9c16744dd8f 
  RESP BODY: 404 Not Found

  The resource could not be found.


  DEBUG (session:248) REQ: curl -g -i -X GET http://ler-cloudnet-01:8774/v2.1/ -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a45b40d5aa2cbf5b76e7e4a0809f5eb4cd5001cb"
  DEBUG (connectionpool:393) "GET /v2.1/ HTTP/1.1" 200 389
  DEBUG (session:277) RESP: [200] Content-Length: 389 X-Compute-Request-Id: req-d2c6d076-2edc-41f2-967d-0b7db72efb17 Vary: X-OpenStack-Nova-API-Version Connection: keep-alive X-Openstack-Nova-Api-Version: 2.1 Date: Thu, 05 May 2016 15:17:35 GMT Content-Type: application/json 
  RESP BODY: {"version": {"status": "CURRENT", "updated": "2013-07-23T11:33:21Z", "links": [{"href": "http://ler-cloudnet-01:8774/v2.1/";, "rel": "self"}, {"href": "http://docs.openstack.org/";, "type": "text/html", "rel": "describedby"}], "min_version": "2.1", "version": "2.25", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2.1"}], "id": "v2.1"}}

  DEBUG (extension:157) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token')
  DEBUG (extension:157) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken')
  DEBUG (extension:157) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode')
  DEBUG (extension:157) found extension EntryPoint.parse('v2password = keystoneauth1.loading._plugins.identity.v2:Password')
  DEBUG (extension:157) found extension EntryPoint.parse('v3password = keystoneauth1.loading._plugins.identity.v3:Password')
  DEBUG (extension:157) found extension EntryPoint.parse('v3oidcpassword = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectPassword')
  DEBUG (extension:157) found extension EntryPoint.parse('token = keystoneauth1.loading._plugins.identity.generic:Token')
  DEBUG (extension:157) found extension EntryPoint.parse('v3token = keystoneauth1.loading._plugins.identity.v3:Token')
  DEBUG (extension:157) found extension EntryPoint.parse('password = keystoneauth1.loading._plugins.identity.generic:Password')
  DEBUG (session:248) REQ: curl -g -i -X GET http://ler-cloudnet-01:8774/v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419/servers/badacdde-0b93-4a96-a790-3c1f6a71ea47 -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.25" -H "X-Auth-Token: {SHA1}a45b40d5aa2cbf5b76e7e4a0809f5eb4cd5001cb"
  DEBUG (connectionpool:393) "GET /v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419/servers/badacdde-0b93-4a96-a790-3c1f6a71ea47 HTTP/1.1" 200 2308
  DEBUG (session:277) RESP: [200] Content-Length: 2308 X-Compute-Request-Id: req-6efd0796-b4f4-4fcc-883b-012af5545f3e Vary: X-OpenStack-Nova-API-Version Connection: keep-alive X-Openstack-Nova-Api-Version: 2.25 Date: Thu, 05 May 2016 15:17:35 GMT Content-Type: application/json 
  RESP BODY: {"server": {"status": "ACTIVE", "OS-EXT-SRV-ATTR:ramdisk_id": "", "updated": "2016-05-05T15:15:43Z", "hostId": "863ae4569a222fdb36b65427b93b537ad801af26001dbd1543f3683a", "OS-EXT-SRV-ATTR:host": "ler-cloudsrv-01", "addresses": {"demo-net": [{"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:19:21:ca", "version": 4, "addr": "192.168.254.11", "OS-EXT-IPS:type": "fixed"}, {"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:19:21:ca", "version": 4, "addr": "172.16.7.103", "OS-EXT-IPS:type": "floating"}]}, "links": [{"href": "http://ler-cloudnet-01:8774/v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419/servers/badacdde-0b93-4a96-a790-3c1f6a71ea47";, "rel": "self"}, {"href": "http://ler-cloudnet-01:8774/32d0fe3c882b4e98b4ad9e3e3dd6e419/servers/badacdde-0b93-4a96-a790-3c1f6a71ea47";, "rel": "bookmark"}], "key_name": "centos", "image": {"id": "ecd8aacb-4089-49f0-bf0e-23572df97b42", "links": [{"href": "http://ler-cloudnet-01:8774/32d0fe3c882b4e98b4ad9e3e3dd6e419/images/ecd8aacb-4089-49f0-bf0e-23572df97b42";, "rel": "bookmark"}]}, "OS-EXT-SRV-ATTR:user_data": null, "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "active", "OS-EXT-SRV-ATTR:instance_name": "instance-00000032", "OS-EXT-SRV-ATTR:root_device_name": "/dev/vda", "OS-SRV-USG:launched_at": "2016-05-02T15:37:30.000000", "locked": false, "flavor": {"id": "d5a0079b-b66b-456d-9077-b75e4116ec04", "links": [{"href": "http://ler-cloudnet-01:8774/32d0fe3c882b4e98b4ad9e3e3dd6e419/flavors/d5a0079b-b66b-456d-9077-b75e4116ec04";, "rel": "bookmark"}]}, "id": "badacdde-0b93-4a96-a790-3c1f6a71ea47", "security_groups": [{"name": "default"}], "description": "vm001", "OS-EXT-SRV-ATTR:kernel_id": "", "host_status": "UP", "OS-EXT-AZ:availability_zone": "nova", "user_id": "b4ecd5437a964c9aa463e630c15b1c5a", "name": "vm001", "OS-EXT-SRV-ATTR:launch_index": 0, "created": "2016-05-02T15:37:18Z", "tenant_id": "1d79170d4847430f8551c28bbd1d4f4d", "OS-DCF:diskConfig": "AUTO", "OS-EXT-SRV-ATTR:hypervisor_hostname": "ler-cloudsrv-01", "os-extended-volumes:volumes_attached": [{"id": "79a4dafd-5f4d-4603-8362-48b63129d000", "delete_on_termination": false}], "accessIPv4": "", "accessIPv6": "", "OS-EXT-SRV-ATTR:reservation_id": "r-40w0zyaa", "OS-EXT-SRV-ATTR:hostname": "vm001", "progress": 0, "OS-EXT-STS:power_state": 1, "config_drive": "", "OS-SRV-USG:terminated_at": null, "metadata": {}}}

  DEBUG (session:248) REQ: curl -g -i -X POST http://ler-cloudnet-01:8774/v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419/servers/badacdde-0b93-4a96-a790-3c1f6a71ea47/action -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.25" -H "X-Auth-Token: {SHA1}a45b40d5aa2cbf5b76e7e4a0809f5eb4cd5001cb" -d '{"os-migrateLive": {"block_migration": "auto", "host": "ler-cloudnet-01"}}'
  DEBUG (connectionpool:393) "POST /v2.1/32d0fe3c882b4e98b4ad9e3e3dd6e419/servers/badacdde-0b93-4a96-a790-3c1f6a71ea47/action HTTP/1.1" 500 216
  DEBUG (session:277) RESP: [500] Content-Length: 216 X-Compute-Request-Id: req-70b112fa-f765-41af-80ba-9e8221dff675 Vary: X-OpenStack-Nova-API-Version Connection: keep-alive X-Openstack-Nova-Api-Version: 2.25 Date: Thu, 05 May 2016 15:18:36 GMT Content-Type: application/json; charset=UTF-8 
  RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'oslo_messaging.exceptions.MessagingTimeout'>", "code": 500}}

  DEBUG (shell:1082) Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-70b112fa-f765-41af-80ba-9e8221dff675)
  Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 1080, in main
      OpenStackComputeShell().main(argv)
    File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 1007, in main
      args.func(self.cs, args)
    File "/usr/lib/python2.7/site-packages/novaclient/v2/shell.py", line 3853, in do_live_migration
      args.block_migrate)
    File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 442, in live_migrate
      block_migration)
    File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 370, in substitution
      return methods[-1].func(obj, *args, **kwargs)
    File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1530, in live_migrate
      'block_migration': block_migration})
    File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1682, in _action
      info=info, **kwargs)
    File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1693, in _action_return_resp_and_body
      return self.api.client.post(url, body=body)
    File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 179, in post
      return self.request(url, 'POST', **kwargs)
    File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 94, in request
      raise exceptions.from_response(resp, body, url, method)
  ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-70b112fa-f765-41af-80ba-9e8221dff675)
  ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-70b112fa-f765-41af-80ba-9e8221dff675)


  [root@ler-cloudnet-01 ~]# yum list installed | grep nova
  openstack-nova-api.noarch       1:13.0.0-1.el7          @centos-openstack-mitaka
  openstack-nova-common.noarch    1:13.0.0-1.el7          @centos-openstack-mitaka
  openstack-nova-compute.noarch   1:13.0.0-1.el7          @centos-openstack-mitaka
  openstack-nova-conductor.noarch 1:13.0.0-1.el7          @centos-openstack-mitaka
  openstack-nova-console.noarch   1:13.0.0-1.el7          @centos-openstack-mitaka
  openstack-nova-novncproxy.noarch
  openstack-nova-scheduler.noarch 1:13.0.0-1.el7          @centos-openstack-mitaka
  python-nova.noarch              1:13.0.0-1.el7          @centos-openstack-mitaka
  python-novaclient.noarch        1:3.3.0-1.el7           @centos-openstack-mitaka

  
  Using Ceph-hammer, kvm, neutron ml2 linux-bridge

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1578691/+subscriptions
Follow ups

[Bug 1578691] Re: live-migration failed Unexpected API Error
From: Markus Zoeller (markus_z), 2016-07-12