yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

I have a feeling this is probably fixed since Pike or at least Rocky
because the DiskFilter has been deprecated in favor of using the
placement service for filtering hosts based on DISK_GB inventory which
accounts for root, ephemeral and swap disk. Furthermore
https://review.opendev.org/#/c/580720/ and a series of changes after it
cleaned up issues with reporting and claiming disk for volume-backed
servers and swap disk.

So I'm going to mark this as Invalid now - if it's still a problem on
master (or the latest release of Stein) then please re-open with new
details.

** Changed in: nova
       Status: In Progress => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1662432

Title:
  Instance storage calculation should use BDM information instead of the
  flavor specification

Status in OpenStack Compute (nova):
  Invalid

Bug description:
  Steps to reproduce
  ==================

  This is the most common case that affected us:

  1. Create flavor A that permits ephemeral storage. For example, an optional 200 GB ephemeral root disk.
  2. Create flavor B that does not permit ephemeral storage.
  3. Ensure that the DiskFilter is configured.
  4. Use flavor A both with and without ephemeral storage in quantity, until the sum of the possible ephemeral storage usage (the flavor specification, not the actual usage, which may be well below the capacity of the machine) exceeds the capacity of the machine. This can eventually be the state of every hypervisor.
  5. Try to schedule a virtual machine with flavor B.

  The other case is where the flavor B permits storage, and the instance
  requests it.

  Expected result
  ===============

  In the case that the flavor B does not permit storage, the DiskFilter
  should permit the scheduler to use any hypervisor whether or not it
  has available local storage, as the flavor does not require or request
  any ephemeral storage, so over commitment of ephemeral storage should
  not be a factor.

  In the case that the flavor B permits storage, and the instance
  requests it, the DiskFilter should permit the scheduler to use any
  hypervisor that has sufficient storage.

  Actual result
  =============

  In both cases, the DiskFilter will prevent the scheduler from using
  any hypervisor where the sum of the possible ephemeral storage usage
  exceeds the capacity of the machine. The virtual machine does not get
  scheduled, and a vague error is provided to the user.

  Environment
  ===========

  Nova 14.0.2 (Newton)
  Libvirt + KVM
  Ephemeral storage is local QCOW2.

  Analysis
  ========

  The DiskFilter relies on the ResourceTracker to calculate storage
  usage on each compute node. The ResourceTracker uses the flavor
  specification. The instances can choose to use less storage than
  the flavor specification permits, and in this case, the calculation
  can be wrong. In extreme cases such as ours, the calculation can be extremely wrong and the "free space" can be a very large negative number.

  To solve the general case, the ResourceTracker should be fixed
  to use the BDM information, and not the flavor specification.

  To be more resistant to usage calculation issues now and in the
  future, I have found an effective work-around to bypass the DiskFilter
  check in the case that no storage is actually requested. From a robust
  code perspective, the usage calculation has been wrong, and it could
  be wrong in the future. Logically, when no storage is requested, the
  DiskFilter should not perform any filter. It could be an acceptable
  compromise here to implement this work-around, but to ensure that it
  will send a warning to the log file to let the operator be aware that
  this work-around is taking effect.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1662432/+subscriptions