yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

Public bug reported:

The rebuild code in the compute manager also handles evacuate. Rebuild
is rebuild on the same host, no migration. Evacuate is rebuild the
instance on another host, and has a migration context.

This code:

https://github.com/openstack/nova/blob/e01ae75d52900d96355dfcb39ef9b136f0c0d5c4/nova/compute/manager.py#L2718

Is using the mutated_migration_context() context manager which lazy-
loads numa_topology, pci_requests and pci_devices and then, since
self.migration_context isn't set on the instance, yields as a noop.

Seen here:

http://logs.openstack.org/82/471082/1/gate/gate-novaclient-dsvm-
functional-neutron-ubuntu-
xenial/796acb7/logs/screen-n-cpu.txt.gz#_Jun_06_13_14_02_547424

Jun 06 13:14:02.547424 ubuntu-xenial-infracloud-chocolate-9158824 nova-compute[20994]: DEBUG nova.objects.instance [None req-5b3770c1-d332-4875-8933-97de8a9890b4 admin admin] Lazy-loading 'pci_requests' on Instance uuid 573258a4-9416-4e13-a765-7c90683f3526 {{(pid=20994) obj_load_attr /opt/stack/new/nova/nova/objects/instance.py:1038}}
Jun 06 13:14:02.562243 ubuntu-xenial-infracloud-chocolate-9158824 nova-compute[20994]: DEBUG nova.objects.instance [None req-5b3770c1-d332-4875-8933-97de8a9890b4 admin admin] Lazy-loading 'pci_devices' on Instance uuid 573258a4-9416-4e13-a765-7c90683f3526 {{(pid=20994) obj_load_attr /opt/stack/new/nova/nova/objects/instance.py:1038}}
Jun 06 13:14:02.577132 ubuntu-xenial-infracloud-chocolate-9158824 nova-compute[20994]: DEBUG nova.objects.instance [None req-5b3770c1-d332-4875-8933-97de8a9890b4 admin admin] Lazy-loading 'migration_context' on Instance uuid 573258a4-9416-4e13-a765-7c90683f3526 {{(pid=20994) obj_load_attr /opt/stack/new/nova/nova/objects/instance.py:1038}}
Jun 06 13:14:02.590554 ubuntu-xenial-infracloud-chocolate-9158824 nova-compute[20994]: DEBUG nova.objects.instance [None req-5b3770c1-d332-4875-8933-97de8a9890b4 admin admin] [instance: 573258a4-9416-4e13-a765-7c90683f3526] Trying to apply a migration context that does not seem to be set for this instance {{(pid=20994) apply_migration_context /opt/stack/new/nova/nova/objects/instance.py:977}}

This is wasteful as each lazy-loaded field is a round trip to the
database via conductor to set the field. If self.migration_context isn't
set, mutated_migration_context() should just yield and return.

** Affects: nova
     Importance: Undecided
         Status: Triaged


** Tags: rebuild

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1696221

Title:
  Unnecessary instance lazy-loads during rebuild

Status in OpenStack Compute (nova):
  Triaged

Bug description:
  The rebuild code in the compute manager also handles evacuate. Rebuild
  is rebuild on the same host, no migration. Evacuate is rebuild the
  instance on another host, and has a migration context.

  This code:

  https://github.com/openstack/nova/blob/e01ae75d52900d96355dfcb39ef9b136f0c0d5c4/nova/compute/manager.py#L2718

  Is using the mutated_migration_context() context manager which lazy-
  loads numa_topology, pci_requests and pci_devices and then, since
  self.migration_context isn't set on the instance, yields as a noop.

  Seen here:

  http://logs.openstack.org/82/471082/1/gate/gate-novaclient-dsvm-
  functional-neutron-ubuntu-
  xenial/796acb7/logs/screen-n-cpu.txt.gz#_Jun_06_13_14_02_547424

  Jun 06 13:14:02.547424 ubuntu-xenial-infracloud-chocolate-9158824 nova-compute[20994]: DEBUG nova.objects.instance [None req-5b3770c1-d332-4875-8933-97de8a9890b4 admin admin] Lazy-loading 'pci_requests' on Instance uuid 573258a4-9416-4e13-a765-7c90683f3526 {{(pid=20994) obj_load_attr /opt/stack/new/nova/nova/objects/instance.py:1038}}
  Jun 06 13:14:02.562243 ubuntu-xenial-infracloud-chocolate-9158824 nova-compute[20994]: DEBUG nova.objects.instance [None req-5b3770c1-d332-4875-8933-97de8a9890b4 admin admin] Lazy-loading 'pci_devices' on Instance uuid 573258a4-9416-4e13-a765-7c90683f3526 {{(pid=20994) obj_load_attr /opt/stack/new/nova/nova/objects/instance.py:1038}}
  Jun 06 13:14:02.577132 ubuntu-xenial-infracloud-chocolate-9158824 nova-compute[20994]: DEBUG nova.objects.instance [None req-5b3770c1-d332-4875-8933-97de8a9890b4 admin admin] Lazy-loading 'migration_context' on Instance uuid 573258a4-9416-4e13-a765-7c90683f3526 {{(pid=20994) obj_load_attr /opt/stack/new/nova/nova/objects/instance.py:1038}}
  Jun 06 13:14:02.590554 ubuntu-xenial-infracloud-chocolate-9158824 nova-compute[20994]: DEBUG nova.objects.instance [None req-5b3770c1-d332-4875-8933-97de8a9890b4 admin admin] [instance: 573258a4-9416-4e13-a765-7c90683f3526] Trying to apply a migration context that does not seem to be set for this instance {{(pid=20994) apply_migration_context /opt/stack/new/nova/nova/objects/instance.py:977}}

  This is wasteful as each lazy-loaded field is a round trip to the
  database via conductor to set the field. If self.migration_context
  isn't set, mutated_migration_context() should just yield and return.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1696221/+subscriptions