yahoo-eng-team team mailing list archive

[OpenStack Infra <1777422@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/587636
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b5b7d86bb04f92d21cf954cd6b3463c9fcc637e6
Submitter: Zuul
Branch:    master

commit b5b7d86bb04f92d21cf954cd6b3463c9fcc637e6
Author: Matt Riedemann <mriedem.os@xxxxxxxxx>
Date:   Tue Jul 31 17:26:47 2018 -0400

    Make ResourceTracker.stats node-specific
    
    As of change I6827137f35c0cb4f9fc4c6f753d9a035326ed01b in
    Ocata, the ResourceTracker manages multiple compute nodes
    via its "compute_nodes" variable, but the "stats" variable
    was still being shared across all nodes, which leads to
    leaking stats across nodes in an ironic deployment where
    a single nova-compute service host is managing multiple
    ironic instances (nodes).
    
    This change makes ResourceTracker.stats node-specific
    which fixes the ironic leak but also allows us to remove
    the stats deepcopy while iterating over instances which
    should improve performance for single-node deployments with
    potentially a large number of instances, i.e. vCenter.
    
    Change-Id: I0b9e5b711878fa47ba90e43c0b41437b57cf8ef6
    Closes-Bug: #1784705
    Closes-Bug: #1777422


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1777422

Title:
  Resource tracker periodic task taking a very long time

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  We have 250 instances on a compute node and the resource tracker
  periodic task is taking very long:

  2018-06-17 10:30:56.194 1658 DEBUG oslo_concurrency.lockutils [req-
  fb2573f9-3862-45db-b546-7a00fdd9a871 - - - - -] Lock
  "compute_resources" released by
  "nova.compute.resource_tracker._update_available_resource" :: held
  10.666s inner /usr/lib/python2.7/dist-
  packages/oslo_concurrency/lockutils.py:288

  This is due to the deepcopy. This copies the structure N times per
  iteration, once for each instance. This is very costly.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1777422/+subscriptions