yahoo-eng-team team mailing list archive

[OpenStack Infra <1716903@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/503601
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=062f5b2e876a09119e43c1905f91610cd4e5d015
Submitter: Jenkins
Branch:    master

commit 062f5b2e876a09119e43c1905f91610cd4e5d015
Author: Yikun Jiang <yikunkero@xxxxxxxxx>
Date:   Wed Sep 13 19:35:49 2017 +0800

    Add @targets_cell for live_migrate_instance method in conductor
    
    With microversion < 2.34, the API casts to the migrate_server
    method in super conductor which targets the context using the
    @targets_cell decorator.
    
    With microversion >= 2.34, the API casts to the live_migrate_instance
    method in super conductor which does not use the @targets_cell
    decorator, which results in a failure to lookup the instance action
    record when recording the start of the action event with the
    @wrap_instance_event decorator.
    
    This change simply adds the decorator and provides a test which
    was missing for this before. Note that the live migration CI job
    didn't catch this regression since it only tests up to microversion
    2.26.
    
    Co-Authored-By: Matt Riedemann <mriedem.os@xxxxxxxxx>
    
    Closes-bug: #1716903
    Change-Id: I21d3f3b7589221b7e0a46c332510afc876ca5a79


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1716903

Title:
  Failed to live-migrate instance in cell with microversion >= 2.34

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) pike series:
  In Progress

Bug description:
  Step 1 create instance in cell1
  +--------------------------------------+--------+--------+------------+-------------+---------------------------------+
  | ID                                   | Name   | Status | Task State | Power State | Networks                        |
  +--------------------------------------+--------+--------+------------+-------------+---------------------------------+
  | 84038890-8d70-45e1-8240-2303f4227e11 | yikun1 | ACTIVE | -          | Running     | public=2001:db8::a, 172.24.4.13 |
  +--------------------------------------+--------+--------+------------+-------------+---------------------------------+

  Step 2 live migrate instance
  nova live-migration 84038890-8d70-45e1-8240-2303f4227e11

  Step 3
  The instance will stuck in "MIGRATIING" state.
  +--------------------------------------+--------+-----------+------------+-------------+---------------------------------+
  | ID                                   | Name   | Status    | Task State | Power State | Networks                        |
  +--------------------------------------+--------+-----------+------------+-------------+---------------------------------+
  | 84038890-8d70-45e1-8240-2303f4227e11 | yikun1 | MIGRATING | migrating  | Running     | public=2001:db8::a, 172.24.4.13 |
  +--------------------------------------+--------+-----------+------------+-------------+---------------------------------+

  It seems we need add @targets_cell decorator for **live_migrate_instance** methods in conductor:
  https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L378

  ERROR LOG in super conductor:
  Exception during message handling: InstanceActionNotFound: Action for request_id req-5aa03558-ae14-458e-9c35-c3d377c7ce45 on instance 84038890-8d70-45e1-8240-2303f4227e11 not found
  Traceback (most recent call last):
    File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
      res = self.dispatcher.dispatch(message)
    File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
      return self._do_dispatch(endpoint, method, ctxt, args)
    File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
      result = func(ctxt, **new_args)
    File "/opt/stack/nova/nova/compute/utils.py", line 875, in decorated_function
      with EventReporter(context, event_name, instance_uuid):
    File "/opt/stack/nova/nova/compute/utils.py", line 846, in __enter__
      self.context, uuid, self.event_name, want_result=False)
    File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
      result = fn(cls, context, *args, **kwargs)
    File "/opt/stack/nova/nova/objects/instance_action.py", line 169, in event_start
      db_event = db.action_event_start(context, values)
    File "/opt/stack/nova/nova/db/api.py", line 1957, in action_event_start
      return IMPL.action_event_start(context, values)
    File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 250, in wrapped
      return f(context, *args, **kwargs)
    File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 6155, in action_event_start
      instance_uuid=values['instance_uuid'])
  InstanceActionNotFound: Action for request_id req-5aa03558-ae14-458e-9c35-c3d377c7ce45 on instance 84038890-8d70-45e1-8240-2303f4227e11 not found

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1716903/+subscriptions