yahoo-eng-team team mailing list archive

[OpenStack Infra <2086675@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/glance/+/934716
Committed: https://opendev.org/openstack/glance/commit/45202b1e044f52aa863ed135564cc1fc74b43145
Submitter: "Zuul (22348)"
Branch:    master

commit 45202b1e044f52aa863ed135564cc1fc74b43145
Author: Abhishek Kekane <akekane@xxxxxxxxxx>
Date:   Tue Nov 12 05:49:23 2024 +0000

    Fix performance glitch while sorting image locations
    
    Some of the available glance stores like file, cinder etc has
    capability to reuse already initiated driver (DRIVER_REUSABLE = 0b01000000). In
    Caracal we have added a feature to sort image locations based on store
    weight. As RBD driver of glance does not have this reuse capability, during
    image list API call it initializes the RBD driver for each of the available
    image which is causing noticable delay in list call.
    
    To avoid this, using new interface added in glance_store which will directly
    get the weight of the store from memory and return it back to user.
    
    Depends-On: https://review.opendev.org/c/openstack/glance_store/+/934362
    Closes-Bug: #2086675
    Change-Id: I662ba19697e03917ca999920ea7be93a0b2a8296


** Changed in: glance
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/2086675

Title:
  Suspected performance regression for RBD back end linked to location
  sorting

Status in Glance:
  Fix Released

Bug description:
  Hi,
  For the past few releases (Caracal, Bobcat, maybe earlier) we have noticed that listing images is particularly slow. I've finally had some time to dig into this, and I believe I've tracked it to https://review.opendev.org/c/openstack/glance/+/886811

  We're running the RBD back end primarily, but also have HTTP and
  Cinder listed in 'enabled_backends', meaning that sort_image_locations
  executes in full. It appears that when this function executes for the
  RBD back end, it causes a connection to be opened to the back end.
  When doing a full image list operation, this happens once for every
  image in the list (the connection is not re-used). This appears to
  carry a 20-30ms time penalty per image. As such, for any reasonable
  set of images the response ends up taking several seconds.

  In our case, images are unlikely to be held in more than one back end
  at a time, and I noted that adding a length check to the locations
  list in
  https://github.com/openstack/glance/blob/master/glance/common/utils.py#L718
  so that the sorting doesn't occur when the list has just one element
  resolves the performance issue entirely.

  Whilst a length check is a workaround, does the sorting operation
  actually require connections to be opened to the RBD back end? If they
  are required, could the connections at least be re-used to avoid this
  time penalty growing linearly with the number of images held by
  Glance?

  Thanks

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/2086675/+subscriptions