yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1709594] [NEW] live-migration without '--block-migrate" failed with "No sql_connection parameter is established"

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Jianghua Wang <wjh_fresh@xxxxxxx>
Date: Wed, 09 Aug 2017 10:40:06 -0000
Reply-to: Bug 1709594 <1709594@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

The test in on XenServer.
nova  live-migration <VM>

If we run the live-migration without the option of "--block-migrate", it will failed with error as:
   RemoteError: Remote error: RemoteError Remote error: CantStartEngineError No sql_connection parameter is established"

- The trace for nova-conductor:
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/opt/stack/nova/nova/conductor/tasks/base.py", line 42, in execute
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     return self._execute()
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/opt/stack/nova/nova/conductor/tasks/live_migrate.py", line 56, in _execute
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     self._check_requested_destination()
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/opt/stack/nova/nova/conductor/tasks/live_migrate.py", line 96, in _check_requested_destination
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     self._call_livem_checks_on_host(self.destination)
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/opt/stack/nova/nova/conductor/tasks/live_migrate.py", line 147, in _call_livem_checks_on_host
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     destination, self.block_migration, self.disk_over_commit)
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/opt/stack/nova/nova/compute/rpcapi.py", line 479, in check_can_live_migrate_destination
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     disk_over_commit=disk_over_commit)
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     retry=self.retry)
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     timeout=timeout, retry=retry)
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     retry=retry)
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     raise result
Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager RemoteError: Remote error: RemoteError Remote error: CantStartEngineError No sql_connection parameter is established

- The trace from nova-compute:
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 211, in decorated_function
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 199, in decorated_function
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 5253, in check_can_live_migrate_destination
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     disk_over_commit)
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 5264, in _do_check_can_live_migrate_destination
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     block_migration, disk_over_commit)
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/xenapi/driver.py", line 465, in check_can_live_migrate_destination
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     disk_over_commit)
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 2281, in check_can_live_migrate_destination
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     self._ensure_host_in_aggregate(ctxt, src)
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 2203, in _ensure_host_in_aggregate
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     self._get_host_uuid_from_aggregate(context, hostname)
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 2189, in _get_host_uuid_from_aggregate
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     aggregate_list = objects.AggregateList.get_by_host(context, CONF.host)
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 177, in wrapper
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     args, kwargs)
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/conductor/rpcapi.py", line 240, in object_class_action_versions
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     args=args, kwargs=kwargs)
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     retry=self.retry)
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     timeout=timeout, retry=retry)
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     retry=retry)
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     raise result
Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server RemoteError: Remote error: CantStartEngineError No sql_connection parameter is established

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1709594

Title:
  live-migration without '--block-migrate" failed with "No
  sql_connection parameter is established"

Status in OpenStack Compute (nova):
  New

Bug description:
  The test in on XenServer.
  nova  live-migration <VM>

  If we run the live-migration without the option of "--block-migrate", it will failed with error as:
     RemoteError: Remote error: RemoteError Remote error: CantStartEngineError No sql_connection parameter is established"

  - The trace for nova-conductor:
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/opt/stack/nova/nova/conductor/tasks/base.py", line 42, in execute
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     return self._execute()
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/opt/stack/nova/nova/conductor/tasks/live_migrate.py", line 56, in _execute
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     self._check_requested_destination()
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/opt/stack/nova/nova/conductor/tasks/live_migrate.py", line 96, in _check_requested_destination
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     self._call_livem_checks_on_host(self.destination)
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/opt/stack/nova/nova/conductor/tasks/live_migrate.py", line 147, in _call_livem_checks_on_host
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     destination, self.block_migration, self.disk_over_commit)
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/opt/stack/nova/nova/compute/rpcapi.py", line 479, in check_can_live_migrate_destination
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     disk_over_commit=disk_over_commit)
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     retry=self.retry)
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     timeout=timeout, retry=retry)
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     retry=retry)
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager     raise result
  Aug 09 03:26:36 DevStackOSDomU nova-conductor[1753]: ERROR nova.conductor.manager RemoteError: Remote error: RemoteError Remote error: CantStartEngineError No sql_connection parameter is established

  - The trace from nova-compute:
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 211, in decorated_function
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     self.force_reraise()
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 199, in decorated_function
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 5253, in check_can_live_migrate_destination
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     disk_over_commit)
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 5264, in _do_check_can_live_migrate_destination
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     block_migration, disk_over_commit)
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/xenapi/driver.py", line 465, in check_can_live_migrate_destination
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     disk_over_commit)
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 2281, in check_can_live_migrate_destination
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     self._ensure_host_in_aggregate(ctxt, src)
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 2203, in _ensure_host_in_aggregate
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     self._get_host_uuid_from_aggregate(context, hostname)
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 2189, in _get_host_uuid_from_aggregate
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     aggregate_list = objects.AggregateList.get_by_host(context, CONF.host)
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 177, in wrapper
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     args, kwargs)
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/conductor/rpcapi.py", line 240, in object_class_action_versions
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     args=args, kwargs=kwargs)
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     retry=self.retry)
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 123, in _send
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     timeout=timeout, retry=retry)
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     retry=retry)
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server     raise result
  Aug 09 03:26:17 DevStackOSDomU nova-compute[1762]: ERROR oslo_messaging.rpc.server RemoteError: Remote error: CantStartEngineError No sql_connection parameter is established

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1709594/+subscriptions