yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1604475] [NEW] TypeError: live_migrate_instance() got an unexpected keyword argument 'async'

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Lauren Taylor <lmtaylor@xxxxxxxxxx>
Date: Tue, 19 Jul 2016 16:19:15 -0000
Reply-to: Bug 1604475 <1604475@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

During live migration, this unexpected API error occur. The live
migration failed.

Error in the nova-api.log:
2016-07-19 02:52:59.734 3825 INFO nova.osapi_compute.wsgi.server [req-7ecad794-1fea-4815-a9d9-5fe2ab829c53 0688b01e6439ca32d698d20789d52169126fb41fb1a4ddafcebb97d854e836c9 a3b6f61367f64e23aad54feb8255f4c1 - - -] 9.109.223.198,127.0.0.1 "GET /v2.1/a3b6f61367f64e23aad54feb8255f4c1/storage-viable-hosts?instance_uuid=5b7cb076-53ca-47b6-891a-95ff00d6dd70 HTTP/1.1" status: 200 len: 443 time: 0.1453900
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions [req-c2bf4983-fc1f-415e-9011-4f626972db87 0688b01e6439ca32d698d20789d52169126fb41fb1a4ddafcebb97d854e836c9 a3b6f61367f64e23aad54feb8255f4c1 - - -] Unexpected exception in API method
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 338, in wrapped
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/migrate_server.py", line 100, in _migrate_live
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     disk_over_commit, host, force)
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 165, in inner
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return function(self, context, instance, *args, **kwargs)
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 172, in _wrapped
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return fn(self, context, instance, *args, **kwargs)
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 146, in inner
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return f(self, context, instance, *args, **kw)
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3322, in live_migrate
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     request_spec=request_spec, async=async)
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions TypeError: live_migrate_instance() got an unexpected keyword argument 'async'
2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions
2016-07-19 02:53:03.766 3827 INFO nova.api.openstack.wsgi [req-c2bf4983-fc1f-415e-9011-4f626972db87 0688b01e6439ca32d698d20789d52169126fb41fb1a4ddafcebb97d854e836c9 a3b6f61367f64e23aad54feb8255f4c1 - - -] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<type 'exceptions.TypeError'>


The issue appears to be caused by this changeset: https://review.openstack.org/#/c/311116/

While the 'async' argument was added to the conductor's
ComputeTaskAPI.live_migrate_instance, the 'async' argument was not added
to the conductor's LocalComputeTaskAPI.live_migrate_instance.


My current Nova config file has this set:

[conductor]
use_local = True

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1604475

Title:
  TypeError: live_migrate_instance() got an unexpected keyword argument
  'async'

Status in OpenStack Compute (nova):
  New

Bug description:
  During live migration, this unexpected API error occur. The live
  migration failed.

  Error in the nova-api.log:
  2016-07-19 02:52:59.734 3825 INFO nova.osapi_compute.wsgi.server [req-7ecad794-1fea-4815-a9d9-5fe2ab829c53 0688b01e6439ca32d698d20789d52169126fb41fb1a4ddafcebb97d854e836c9 a3b6f61367f64e23aad54feb8255f4c1 - - -] 9.109.223.198,127.0.0.1 "GET /v2.1/a3b6f61367f64e23aad54feb8255f4c1/storage-viable-hosts?instance_uuid=5b7cb076-53ca-47b6-891a-95ff00d6dd70 HTTP/1.1" status: 200 len: 443 time: 0.1453900
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions [req-c2bf4983-fc1f-415e-9011-4f626972db87 0688b01e6439ca32d698d20789d52169126fb41fb1a4ddafcebb97d854e836c9 a3b6f61367f64e23aad54feb8255f4c1 - - -] Unexpected exception in API method
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions Traceback (most recent call last):
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 338, in wrapped
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/migrate_server.py", line 100, in _migrate_live
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     disk_over_commit, host, force)
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 165, in inner
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return function(self, context, instance, *args, **kwargs)
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 172, in _wrapped
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return fn(self, context, instance, *args, **kwargs)
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 146, in inner
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     return f(self, context, instance, *args, **kw)
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3322, in live_migrate
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions     request_spec=request_spec, async=async)
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions TypeError: live_migrate_instance() got an unexpected keyword argument 'async'
  2016-07-19 02:53:03.764 3827 ERROR nova.api.openstack.extensions
  2016-07-19 02:53:03.766 3827 INFO nova.api.openstack.wsgi [req-c2bf4983-fc1f-415e-9011-4f626972db87 0688b01e6439ca32d698d20789d52169126fb41fb1a4ddafcebb97d854e836c9 a3b6f61367f64e23aad54feb8255f4c1 - - -] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <type 'exceptions.TypeError'>

  
  The issue appears to be caused by this changeset: https://review.openstack.org/#/c/311116/

  While the 'async' argument was added to the conductor's
  ComputeTaskAPI.live_migrate_instance, the 'async' argument was not
  added to the conductor's LocalComputeTaskAPI.live_migrate_instance.


  My current Nova config file has this set:

  [conductor]
  use_local = True

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1604475/+subscriptions
Follow ups

[Bug 1604475] Re: TypeError: live_migrate_instance() got an unexpected keyword argument 'async'
From: OpenStack Infra, 2016-07-19