yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1707934] [NEW] RequestSpec not found error message during VM migration

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Marcus Klein <kleini76@xxxxxxxxx>
Date: Tue, 01 Aug 2017 14:37:45 -0000
Reply-to: Bug 1707934 <1707934@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

When migrating VMs I get the following error message in Nova API logs
with the Newton release:

Unexpected exception in API method
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 338, in wrapped
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/migrate_server.py", line 52, in _migrate
    self.compute_api.resize(req.environ['nova.context'], instance)
  File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 166, in inner
    return function(self, context, instance, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 173, in _wrapped
    return fn(self, context, instance, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 147, in inner
    return f(self, context, instance, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 3033, in resize
    request_spec=request_spec)
  File "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 192, in resize_instance
    request_spec=request_spec)
  File "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 317, in migrate_server
    return cctxt.call(context, 'migrate_server', **kw)
  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call
    retry=self.retry)
  File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 97, in _send
    timeout=timeout, retry=retry)
  File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 464, in send
    retry=retry)
  File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 455, in _send
    raise result
RequestSpecNotFound_Remote: RequestSpec not found for instance 6415561e-d93e-4497-bec4-572aa24544a9
Traceback (most recent call last):

  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
    res = self.dispatcher.dispatch(message)

  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
    return self._do_dispatch(endpoint, method, ctxt, args)

  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
    result = func(ctxt, **new_args)

  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 199, in inner
    return func(*args, **kwargs)

  File "/usr/lib/python2.7/dist-packages/nova/compute/utils.py", line 613, in decorated_function
    return function(self, context, *args, **kwargs)

  File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 220, in migrate_server
    reservations, clean_shutdown, request_spec)

  File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 287, in _cold_migrate
    request_spec.save()

  File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper
    return fn(self, *args, **kwargs)

  File "/usr/lib/python2.7/dist-packages/nova/objects/request_spec.py", line 484, in save
    db_spec = self._save_in_db(self._context, self.instance_uuid, updates)

  File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 824, in wrapper
    return fn(*args, **kwargs)

  File "/usr/lib/python2.7/dist-packages/nova/objects/request_spec.py", line 476, in _save_in_db
    instance_uuid)

  File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 824, in wrapper
    return fn(*args, **kwargs)

  File "/usr/lib/python2.7/dist-packages/nova/objects/request_spec.py", line 425, in _get_by_instance_uuid_from_db
    instance_uuid=instance_uuid)

RequestSpecNotFound: RequestSpec not found for instance 6415561e-
d93e-4497-bec4-572aa24544a9

HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'RequestSpecNotFound_Remote'>
192.168.32.26 "POST /v2.1/d7400e965d5a40a79708649ae64a9d85/servers/6415561e-d93e-4497-bec4-572aa24544a9/action HTTP/1.1" status: 500 len: 567 time: 0.5891929

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1707934

Title:
  RequestSpec not found error message during VM migration

Status in OpenStack Compute (nova):
  New

Bug description:
  When migrating VMs I get the following error message in Nova API logs
  with the Newton release:

  Unexpected exception in API method
  Traceback (most recent call last):
    File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 338, in wrapped
      return f(*args, **kwargs)
    File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/migrate_server.py", line 52, in _migrate
      self.compute_api.resize(req.environ['nova.context'], instance)
    File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 166, in inner
      return function(self, context, instance, *args, **kwargs)
    File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 173, in _wrapped
      return fn(self, context, instance, *args, **kwargs)
    File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 147, in inner
      return f(self, context, instance, *args, **kw)
    File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 3033, in resize
      request_spec=request_spec)
    File "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 192, in resize_instance
      request_spec=request_spec)
    File "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 317, in migrate_server
      return cctxt.call(context, 'migrate_server', **kw)
    File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call
      retry=self.retry)
    File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 97, in _send
      timeout=timeout, retry=retry)
    File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 464, in send
      retry=retry)
    File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 455, in _send
      raise result
  RequestSpecNotFound_Remote: RequestSpec not found for instance 6415561e-d93e-4497-bec4-572aa24544a9
  Traceback (most recent call last):

    File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
      res = self.dispatcher.dispatch(message)

    File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
      return self._do_dispatch(endpoint, method, ctxt, args)

    File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
      result = func(ctxt, **new_args)

    File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 199, in inner
      return func(*args, **kwargs)

    File "/usr/lib/python2.7/dist-packages/nova/compute/utils.py", line 613, in decorated_function
      return function(self, context, *args, **kwargs)

    File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 220, in migrate_server
      reservations, clean_shutdown, request_spec)

    File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 287, in _cold_migrate
      request_spec.save()

    File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper
      return fn(self, *args, **kwargs)

    File "/usr/lib/python2.7/dist-packages/nova/objects/request_spec.py", line 484, in save
      db_spec = self._save_in_db(self._context, self.instance_uuid, updates)

    File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 824, in wrapper
      return fn(*args, **kwargs)

    File "/usr/lib/python2.7/dist-packages/nova/objects/request_spec.py", line 476, in _save_in_db
      instance_uuid)

    File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 824, in wrapper
      return fn(*args, **kwargs)

    File "/usr/lib/python2.7/dist-packages/nova/objects/request_spec.py", line 425, in _get_by_instance_uuid_from_db
      instance_uuid=instance_uuid)

  RequestSpecNotFound: RequestSpec not found for instance 6415561e-
  d93e-4497-bec4-572aa24544a9

  HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'RequestSpecNotFound_Remote'>
  192.168.32.26 "POST /v2.1/d7400e965d5a40a79708649ae64a9d85/servers/6415561e-d93e-4497-bec4-572aa24544a9/action HTTP/1.1" status: 500 len: 567 time: 0.5891929

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1707934/+subscriptions
Follow ups

[Bug 1707934] Re: RequestSpec not found error message during VM migration
From: Marcus Klein, 2017-08-02