yahoo-eng-team team mailing list archive

[OpenStack Infra <1467927@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/229575
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=d5bed8fad9d51dd0d4ba9cf452c6f5c8c543b264
Submitter: Jenkins
Branch:    master

commit d5bed8fad9d51dd0d4ba9cf452c6f5c8c543b264
Author: Nikola Dipanov <ndipanov@xxxxxxxxxx>
Date:   Wed Sep 30 19:30:59 2015 +0100

    Fix CPU pinning for odd number of CPUs w hyperthreading
    
    Previous logic was not handling the cases when the instance cell has an
    odd number of vcpus that is larger than any sibling set on the host. The
    logic would (accidentally) either reject to pin such an instance even
    though there is ample free cores, or would (in case there were enough
    free sibling sets on the host) spread the instance instead of pack it
    onto siblings as the default policy suggests.
    
    This patch fixes some incorrect assumptions in the code, while also
    simplifying it. As an added bonus - we still attempt to expose
    (via the Topology, and this time correctly) the largest possible number
    of threads that we can expose to the instance.
    
    Finally - we add some more comments to clear up the intent behind the
    current packing logic with pointers how it could be tweaked to achieve
    different results in the future.
    
    Change-Id: I2c0b3b250ffb1a7483299df13b317cdb24f8141d
    Co-Authored-By: Stephen Finucane <stephen.finucane@xxxxxxxxx>
    Closes-bug: 1501358
    Closes-bug: 1467927


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1467927

Title:
  Odd number of vCPUs breaks 'prefer' threads policy

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  Using a CPU policy of dedicated ('hw:cpu_policy=dedicated') results in
  vCPUs being pinned to pCPUs, per the original blueprint:

      http://specs.openstack.org/openstack/nova-
  specs/specs/kilo/implemented/virt-driver-cpu-pinning.html

  When scheduling instance with this extra spec there appears to be an
  implicit use of the 'prefer' threads policy, i.e. where possible vCPUs
  are pinned to thread siblings first. This is "implicit" because the
  threads policy aspect of this spec has not yet been implemented.

  However, this implicit 'prefer' policy breaks when a VM with an odd
  number of vCPUs is booted. This has been seen on a Hyper-Threading-
  enabled host where "sibling sets" are two long, but it would
  presumably happen on any host where the number of siblings (or any
  number between this value and one) is not an factor of the number of
  vCPUs (i.e. vCPUs % n != 0, for siblings <= n > 0).

  It is reasonable to assume that a three vCPU VM, for example, should
  try best effort and use siblings for at the first two vCPUs of the VM
  (assuming you're on a host system with HyperThreading and sibling sets
  are of length two). This would give us a true best effort
  implementation.

  ---

  # Testing Configuration

  Testing was conducted on a single-node, Fedora 21-based
  (3.17.8-300.fc21.x86_64) OpenStack instance (built with devstack). The
  system is a dual-socket, 10 core, HT-enabled system (2 sockets * 10
  cores * 2 threads = 40 "pCPUs". 0-9,20-29 = node0, 10-19,30-39 =
  node1). Two flavors were used:

      openstack flavor create --ram 4096 --disk 20 --vcpus 3 demo.odd
      nova flavor-key demo.odd set hw:cpu_policy=dedicated

      openstack flavor create --ram 4096 --disk 20 --vcpus 4 demo.even
      nova flavor-key demo.even set hw:cpu_policy=dedicated

  # Results

  Correct case ("even" number of vCPUs)
  =====================================

  The output from 'virsh dumpxml [ID]' for the four vCPU VM is given
  below. Similar results can be seen for varying "even" numbers of vCPUs
  (2, 4, 10 tested):

      <cputune>
          <shares>4096</shares>
          <vcpupin vcpu='0' cpuset='3'/>
          <vcpupin vcpu='1' cpuset='23'/>
          <vcpupin vcpu='2' cpuset='26'/>
          <vcpupin vcpu='3' cpuset='6'/>
          <emulatorpin cpuset='3,6,23,26'/>
      </cputune>

  Incorrect case ("odd" number of vCPUs)
  ======================================

  The output from 'virsh dumpxml [ID]' for the three vCPU VM is given
  below. Similar results can be seen for varying "odd" numbers of vCPUs
  (3, 5 tested):

      <cputune>
          <shares>3072</shares>
          <vcpupin vcpu='0' cpuset='1'/>
          <vcpupin vcpu='1' cpuset='0'/>
          <vcpupin vcpu='2' cpuset='25'/>
          <emulatorpin cpuset='0-1,25'/>
      </cputune>

  This isn't correct. We would expect something closer to this:

      <cputune>
          <shares>3072</shares>
          <vcpupin vcpu='0' cpuset='0'/>
          <vcpupin vcpu='1' cpuset='20'/>
          <vcpupin vcpu='2' cpuset='1'/>
          <emulatorpin cpuset='0-1,20'/>
      </cputune>

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1467927/+subscriptions