yahoo-eng-team team mailing list archive

[OpenStack Infra <1785827@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/589567
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e6af812865553fbc49114a419170693ff15d5545
Submitter: Zuul
Branch:    master

commit e6af812865553fbc49114a419170693ff15d5545
Author: Lee Yarwood <lyarwood@xxxxxxxxxx>
Date:   Thu Aug 9 13:45:13 2018 +0100

    libvirt: Use os.stat and os.path.getsize for RAW disk inspection
    
    At present when inspecting a file based image we always use ``qemu-img`` to
    determine the virtual size of the image. This works well but can lead to
    the resource tracker taking considerable time to update on hosts with
    a large number of instances/images.
    
    This change switches to using os.stat and os.path.getsize to determine
    the allocated and virtual disk sizes of RAW disks.
    
    Future changes will look into caching the virtual size of the disk
    within disk.info locally on the host to also improve this for qcow2 and
    ploop, further simplifying this code path.
    
    Closes-bug: #1785827
    Change-Id: Ic5c41493dcdcd807209be2beaae0dbbdf5d2ba3f


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1785827

Title:
  Performance regression in libvirt get_available_resource()

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  Description
  ===========
  Periodic task update_available_resource takes 20 seconds for 100
  instances on local storage:

  https://github.com/openstack/nova/blob/stable/pike/nova/compute/resource_tracker.py#L694

  Steps to reproduce
  ==================
  1. /etc/nova/nova.conf:
  [DEFAULT]
  preallocate_images=space
  [libvirt]
  images_type=raw

  2. Launch 100 instances on the host

  3. See, that every update_available_resource() takes 20 seconds

  This performance regression was introduced in this commit:
  https://github.com/openstack/nova/commit/d88b75e81eabfbd463007f6a4f27e6966a466530
  and following commit doubles the time:
  https://github.com/openstack/nova/commit/938c0a745325fa73d098c6d5ddd20b2a599f9624

  Expected result
  ===============
  update_available_resource() takes less than 5 seconds

  Actual result
  =============
  update_available_resource() lasts for 20-30 seconds

  Environment
  ===========
  1. Exact version of OpenStack you are running:
  OpenStack Pike
  nova 16.1.4, commit b58c7f0

  2. Which hypervisor did you use?
     Libvirt + KVM/QEMU

  2. Which storage type did you use?
     local storage, raw

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1785827/+subscriptions