yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

The reason the live migration CI job didn't catch this is that it
doesn't test microversion >= 2.34 which changes to use the
live_migrate_instance method. If the microversion < 2.34 it uses the
migrate_server method which has the @targets_cell decorator.

** Summary changed:

- Failed to live-migrate instance in cell.
+ Failed to live-migrate instance in cell with microversion >= 2.34

** Changed in: nova
   Importance: Undecided => High

** Also affects: nova/pike
   Importance: Undecided
       Status: New

** Changed in: nova/pike
       Status: New => Confirmed

** Changed in: nova/pike
   Importance: Undecided => High

** Tags added: cells live-migration

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1716903

Title:
  Failed to live-migrate instance in cell with microversion >= 2.34

Status in OpenStack Compute (nova):
  In Progress
Status in OpenStack Compute (nova) pike series:
  Confirmed

Bug description:
  Step 1 create instance in cell1
  +--------------------------------------+--------+--------+------------+-------------+---------------------------------+
  | ID                                   | Name   | Status | Task State | Power State | Networks                        |
  +--------------------------------------+--------+--------+------------+-------------+---------------------------------+
  | 84038890-8d70-45e1-8240-2303f4227e11 | yikun1 | ACTIVE | -          | Running     | public=2001:db8::a, 172.24.4.13 |
  +--------------------------------------+--------+--------+------------+-------------+---------------------------------+

  Step 2 live migrate instance
  nova live-migration 84038890-8d70-45e1-8240-2303f4227e11

  Step 3
  The instance will stuck in "MIGRATIING" state.
  +--------------------------------------+--------+-----------+------------+-------------+---------------------------------+
  | ID                                   | Name   | Status    | Task State | Power State | Networks                        |
  +--------------------------------------+--------+-----------+------------+-------------+---------------------------------+
  | 84038890-8d70-45e1-8240-2303f4227e11 | yikun1 | MIGRATING | migrating  | Running     | public=2001:db8::a, 172.24.4.13 |
  +--------------------------------------+--------+-----------+------------+-------------+---------------------------------+

  It seems we need add @targets_cell decorator for **live_migrate_instance** methods in conductor:
  https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L378

  ERROR LOG in super conductor:
  Exception during message handling: InstanceActionNotFound: Action for request_id req-5aa03558-ae14-458e-9c35-c3d377c7ce45 on instance 84038890-8d70-45e1-8240-2303f4227e11 not found
  Traceback (most recent call last):
    File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
      res = self.dispatcher.dispatch(message)
    File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
      return self._do_dispatch(endpoint, method, ctxt, args)
    File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
      result = func(ctxt, **new_args)
    File "/opt/stack/nova/nova/compute/utils.py", line 875, in decorated_function
      with EventReporter(context, event_name, instance_uuid):
    File "/opt/stack/nova/nova/compute/utils.py", line 846, in __enter__
      self.context, uuid, self.event_name, want_result=False)
    File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
      result = fn(cls, context, *args, **kwargs)
    File "/opt/stack/nova/nova/objects/instance_action.py", line 169, in event_start
      db_event = db.action_event_start(context, values)
    File "/opt/stack/nova/nova/db/api.py", line 1957, in action_event_start
      return IMPL.action_event_start(context, values)
    File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 250, in wrapped
      return f(context, *args, **kwargs)
    File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 6155, in action_event_start
      instance_uuid=values['instance_uuid'])
  InstanceActionNotFound: Action for request_id req-5aa03558-ae14-458e-9c35-c3d377c7ce45 on instance 84038890-8d70-45e1-8240-2303f4227e11 not found

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1716903/+subscriptions