yahoo-eng-team team mailing list archive

[OpenStack Infra <1846777@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/686835
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=2d84a564fbb528734776d8432a31a4bb4b40af71
Submitter: Zuul
Branch:    master

commit 2d84a564fbb528734776d8432a31a4bb4b40af71
Author: Matt Riedemann <mriedem.os@xxxxxxxxx>
Date:   Fri Oct 4 16:32:33 2019 -0400

    Cache image GETs for multi-create/multi-BDM requests
    
    This builds on change I1233fbbaebd1750f572432bf74759b94408a17e3 which
    added a functional test to create 10 servers with 255 BDMs each using
    the two images. That scenario results in 2551 calls to the image service
    to get the images. While an unrealistic scenario, it shows how
    inefficient this can be even in a simple multi-create request where
    each server has one or more BDMs that are using one or more of the
    same images.
    
    This change adds a simple cache that is used while processing the list
    of instances to create and the list of BDMs per instance that are using
    a source_type of 'image' to validate the image. This drops the image GET
    calls in the functional test from 2551 to 3.
    
    The functional test verifies the cache is being used properly. Existing
    unit tests are updated for the method signature changes. Note that some
    API controller tests were mocking a bit too deep and ignoring all of the
    parameters to the _validate_bdm mock so rather than continue that pattern
    the assertions are just changed to assert_called_once() since the tests
    clearly don't care about the actual parameters passed to that method.
    
    Change-Id: Ibd51d6fc2d4c9fd142ba17f2f87214bb62f72440
    Closes-Bug: #1846777


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1846777

Title:
  Inefficient/redundant image GETs during large boot from volume server
  create requests with the same image

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  This is demonstrated by this functional test patch:

  https://review.opendev.org/#/c/686734/

  That adds a test which creates a single server create request to
  create 10 servers and each server has 255 BDMs using the same image
  and asserts that the API calls GET /v2/images/{image_id} on the same
  image 2551 times which is pretty inefficient.

  For the lifetime of the server create request we should be smarter and
  cache the results of each image we get so we don't make the same
  redundant calls to the image service.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1846777/+subscriptions