yahoo-eng-team team mailing list archive

[Mikhail Chernik <mchernik@xxxxxxxxxxxx>]

Public bug reported:

The 'require' CPU thread policy allows booting instances on hardware
threads siblings only. However, if a compute host has several NUMA nodes
and at least one NUMA node does not contribute to a list of pinned CPU
cores, NUMATopologyFilter excludes such node from possible targets to an
instance.

Steps to reproduce:

1) On a compute with several NUMA nodes vcpu_pin_set key of nova.conf
does not contain sibling thread cores from one NUMA node (e.g. they are
allocated for OVS with DPDK support)

ex: vcpu_pin_set=0,1,20,21

List of cores on a NUMA node can be obtained with lstopo or numactl -H command: 
# numactl -H
available: 4 nodes (0-3)
node 0 cpus: 0 1 2 3 4 20 21 22 23 24
node 0 size: 64328 MB
node 0 free: 21952 MB
node 1 cpus: 5 6 7 8 9 25 26 27 28 29
node 1 size: 64510 MB
node 1 free: 30091 MB

Sibling cores can be found with # cat
/sys/bus/cpu/devices/cpu*/topology/thread_siblings_list command

# cat /sys/bus/cpu/devices/cpu{0,1}/topology/thread_siblings_list 
0,20
1,21

2) Create corresponding flavor

# openstack flavor create pinned.threads_require --ram 2048 --disk 0 --vcpus 2
# openstack flavor set pinned.threads_require --property "hw:cpu_thread_policy=require" --property "hw:cpu_policy=dedicated"

3) Boot an instance with created flavor
# openstack server create --flavor pinned.threads_require --image cirros vm1

4) List instances:
# openstack server list

Expected result: 
Instance vm1 is booted

Actiual result:
Instance vm1 is in ERROR state with message 'No valid host was found. There are not enough hosts available.' (assuming there are no other computes with sibling thread cores available)

** Affects: nova
     Importance: Undecided
     Assignee: Mikhail Chernik (mchernik)
         Status: In Progress

** Changed in: nova
       Status: New => In Progress

** Changed in: nova
     Assignee: (unassigned) => Mikhail Chernik (mchernik)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1596692

Title:
  NUMATopologyFilter filters out a compute from candidates for booting a
  VM with  'hw:cpu_thread_policy=require' unless cores from all NUMA
  nodes are enabled in vcpu_pin_set on the compute

Status in OpenStack Compute (nova):
  In Progress

Bug description:
  The 'require' CPU thread policy allows booting instances on hardware
  threads siblings only. However, if a compute host has several NUMA
  nodes and at least one NUMA node does not contribute to a list of
  pinned CPU cores, NUMATopologyFilter excludes such node from possible
  targets to an instance.

  Steps to reproduce:

  1) On a compute with several NUMA nodes vcpu_pin_set key of nova.conf
  does not contain sibling thread cores from one NUMA node (e.g. they
  are allocated for OVS with DPDK support)

  ex: vcpu_pin_set=0,1,20,21

  List of cores on a NUMA node can be obtained with lstopo or numactl -H command: 
  # numactl -H
  available: 4 nodes (0-3)
  node 0 cpus: 0 1 2 3 4 20 21 22 23 24
  node 0 size: 64328 MB
  node 0 free: 21952 MB
  node 1 cpus: 5 6 7 8 9 25 26 27 28 29
  node 1 size: 64510 MB
  node 1 free: 30091 MB

  Sibling cores can be found with # cat
  /sys/bus/cpu/devices/cpu*/topology/thread_siblings_list command

  # cat /sys/bus/cpu/devices/cpu{0,1}/topology/thread_siblings_list 
  0,20
  1,21

  2) Create corresponding flavor

  # openstack flavor create pinned.threads_require --ram 2048 --disk 0 --vcpus 2
  # openstack flavor set pinned.threads_require --property "hw:cpu_thread_policy=require" --property "hw:cpu_policy=dedicated"

  3) Boot an instance with created flavor
  # openstack server create --flavor pinned.threads_require --image cirros vm1

  4) List instances:
  # openstack server list

  Expected result: 
  Instance vm1 is booted

  Actiual result:
  Instance vm1 is in ERROR state with message 'No valid host was found. There are not enough hosts available.' (assuming there are no other computes with sibling thread cores available)

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1596692/+subscriptions