yahoo-eng-team team mailing list archive

[OpenStack Infra <1550269@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/285232
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ad0047e97b2847412ee28bad6a3bfb48395add35
Submitter: Jenkins
Branch:    master

commit ad0047e97b2847412ee28bad6a3bfb48395add35
Author: Stephen Finucane <stephen.finucane@xxxxxxxxx>
Date:   Fri Feb 26 10:55:55 2016 +0000

    virt/hardware: Check for threads when "required"
    
    The 'require' case "requires" the presence of hardware threads on a
    host. At present, this check is done using the NUMATopology filter.
    Unfortunately, this means that if this filter is disabled then
    instances can be scheduled on invalid hosts. Resolve this by adding a
    new check to be run when hosts are actually scheduling.
    
    Change-Id: Ia9e4784e02ca9ce7a3d81c962b95bee100f6db42
    Closes-bug: #1550269


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1550269

Title:
  'hw:cpu_thread_policy=require' does not function correctly if
  NUMATopologyFilter is disabled

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  The 'require' policy is supposed to restrict instances to hosts that
  support hardware threads, e.g. HyperThreading. However, this filtering
  is done as part of the NUMATopologyFilter. If this filter is disabled,
  the host boots just fine. This should not be the case and needs to be
  fixed. Findings below.

  ---

  Testing was conducted on a single-node, Fedora 23-based (4.3.5-300.fc23.x86_64)
  OpenStack instance (built with devstack). The system is a dual-socket, ten core,
  HT-enabled system (2 sockets * 10 cores * 2 threads = 40 "pCPUs".
  0-9,20-29 = node0, 10-19,30-39 = node1).

  Commit '8bafc9' of Nova was used.

  # Steps

  ## Create flavors

      $ openstack flavor create pinned.require \
          --id 102 --ram 2048 --disk 0 --vcpus 4
      $ openstack flavor set pinned.require \
          --property "hw:cpu_policy=dedicated" \
          --property "hw:cpu_thread_policy=require"

  ## Validate a HT-enabled node

  The 'require' case is a stricter version of the `prefer` case, in that it
  should fail if we have HyperThreading disabled, do not have enough free
  sibling sets, or have no HyperThreading support at all. However, since we're
  not hitting any of these conditions on this host, things should function just
  like they do for the `prefer` case. Therefore, the guest should see a two
  sockets with one core per socket and two threads per core.

      $ openstack server create --flavor=pinned.require \
          --image=cirros-0.3.4-x86_64-uec --wait test1

      $ sudo virsh list
       Id    Name                           State
      ----------------------------------------------------
       2     instance-00000002              running

      $ sudo virsh dumpxml 2
      <domain type='kvm' id='2'>
        <name>instance-00000002</name>
        ...
        <vcpu placement='static'>4</vcpu>
        <cputune>
          <shares>4096</shares>
          <vcpupin vcpu='0' cpuset='1'/>
          <vcpupin vcpu='1' cpuset='21'/>
          <vcpupin vcpu='2' cpuset='0'/>
          <vcpupin vcpu='3' cpuset='20'/>
          <emulatorpin cpuset='0-1,20-21'/>
        </cputune>
        <numatune>
          <memory mode='strict' nodeset='0'/>
          <memnode cellid='0' mode='strict' nodeset='0'/>
        </numatune>
        ...
        <cpu>
          <topology sockets='2' cores='1' threads='2'/>
          <numa>
            <cell id='0' cpus='0-3' memory='2097152' unit='KiB'/>
          </numa>
        </cpu>
        ...
      </domain>

      $ openstack server delete test1

  No issues here.

  ## Validate a HT-disabled node

  This policy "requires" HyperThreading or similar on the host, so it shouldn't
  work here.

      $ openstack server create --flavor=pinned.require \
          --image=cirros-0.3.4-x86_64-uec --wait test1

      $ sudo virsh list
       Id    Name                           State
      ----------------------------------------------------
       2     instance-00000002              running

      $ sudo virsh dumpxml 2
      <domain type='kvm' id='2'>
        <name>instance-00000002</name>
        ...
        <vcpu placement='static'>4</vcpu>
        <cputune>
          <shares>4096</shares>
          <vcpupin vcpu='0' cpuset='0'/>
          <vcpupin vcpu='1' cpuset='1'/>
          <vcpupin vcpu='2' cpuset='2'/>
          <vcpupin vcpu='3' cpuset='3'/>
          <emulatorpin cpuset='0-3'/>
        </cputune>
        <numatune>
          <memory mode='strict' nodeset='0'/>
          <memnode cellid='0' mode='strict' nodeset='0'/>
        </numatune>
        ...
        <cpu>
          <topology sockets='2' cores='1' threads='2'/>
          <numa>
            <cell id='0' cpus='0-3' memory='2097152' unit='KiB'/>
          </numa>
        </cpu>
        ...
      </domain>

      $ openstack server delete test1

  This is a problem, but we do not currently have the filter activated:

      $ cat /etc/nova/nova.conf | grep scheduler_default_filters
      scheduler_default_filters = RetryFilter,AvailabilityZoneFilter,RamFilter,\
      DiskFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,\
      ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,SameHostFilter,\
      DifferentHostFilter

  Let's activate this:

      $ cat /etc/nova/nova.conf | grep scheduler_default_filters
      scheduler_default_filters = RetryFilter,AvailabilityZoneFilter,RamFilter,\
      DiskFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,\
      ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,SameHostFilter,\
      DifferentHostFilter,NUMATopologyFilter

  And try again:

      $ openstack server create --flavor=pinned.require \
          --image=cirros-0.3.4-x86_64-uec --wait test1
      Error creating server: test1

      Error creating server

  That's more like it, but it shouldn't be necessary.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1550269/+subscriptions