yahoo-eng-team team mailing list archive

[Ricardo Noriega <rnoriega@xxxxxxxxxx>]

** Also affects: nova
   Importance: Undecided
       Status: New

** No longer affects: nova-solver-scheduler

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1578150

Title:
  hw:cpu_thread_policy=isolated allows allocation of siblings

Status in OpenStack Compute (nova):
  New

Bug description:
  According to description of cpu_thread_policy:

   ``isolate``: The host must not have an SMT architecture or must emulate
      a non-SMT architecture. If the host does not have an SMT architecture,
      each vCPU is placed on a different core as expected. If the host does
     have an SMT architecture - that is, one or more cores have thread
     siblings - then each vCPU is placed on a different physical core. No
     vCPUs from other guests are placed on the same core. All but one thread
      sibling on each utilized core is therefore guaranteed to be unusable.

  Having 20 threads available to allocate 10 isolated vCPUs:

  nova flavor-create isolated auto 1024 10 5
  nova flavor-key isolated set hw:cpu_policy=dedicated
  nova flavor-key isolated set hw:cpu_thread_policy=isolate
  nova flavor-key isolated set hw:numa_nodes=1
  nova boot testIso1 --flavor isolated --image cirros --nic net-id=$NET_ID
  nova boot testIso2 --flavor isolated --image cirros --nic net-id=$NET_ID

  
  # virsh vcpupin 2
  VCPU: CPU Affinity
  ----------------------------------
     0: 10
     1: 16
     2: 8
     3: 18
     4: 2

  
  # virsh vcpupin 3
  VCPU: CPU Affinity
  ----------------------------------
     0: 17
     1: 3
     2: 11
     3: 9
     4: 19

  
  Cheking CPU topology:

  NUMANode L#0 (P#0 32GB)
      Socket L#0 + L3 L#0 (15MB)
        L2 L#0 (256KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0
          PU L#0 (P#0)
          PU L#1 (P#12)
        L2 L#1 (256KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1
          PU L#2 (P#2)
          PU L#3 (P#14)
        L2 L#2 (256KB) + L1d L#2 (32KB) + L1i L#2 (32KB) + Core L#2
          PU L#4 (P#4)
          PU L#5 (P#16)
        L2 L#3 (256KB) + L1d L#3 (32KB) + L1i L#3 (32KB) + Core L#3
          PU L#6 (P#6)
          PU L#7 (P#18)
        L2 L#4 (256KB) + L1d L#4 (32KB) + L1i L#4 (32KB) + Core L#4
          PU L#8 (P#8)
          PU L#9 (P#20)
        L2 L#5 (256KB) + L1d L#5 (32KB) + L1i L#5 (32KB) + Core L#5
          PU L#10 (P#10)
          PU L#11 (P#22)

    NUMANode L#1 (P#1 32GB) + Socket L#1 + L3 L#1 (15MB)
      L2 L#6 (256KB) + L1d L#6 (32KB) + L1i L#6 (32KB) + Core L#6
        PU L#12 (P#1)
        PU L#13 (P#13)
      L2 L#7 (256KB) + L1d L#7 (32KB) + L1i L#7 (32KB) + Core L#7
        PU L#14 (P#3)
        PU L#15 (P#15)
      L2 L#8 (256KB) + L1d L#8 (32KB) + L1i L#8 (32KB) + Core L#8
        PU L#16 (P#5)
        PU L#17 (P#17)
      L2 L#9 (256KB) + L1d L#9 (32KB) + L1i L#9 (32KB) + Core L#9
        PU L#18 (P#7)
        PU L#19 (P#19)
      L2 L#10 (256KB) + L1d L#10 (32KB) + L1i L#10 (32KB) + Core L#10
        PU L#20 (P#9)
        PU L#21 (P#21)
      L2 L#11 (256KB) + L1d L#11 (32KB) + L1i L#11 (32KB) + Core L#11
        PU L#22 (P#11)
        PU L#23 (P#23)

  Now, we have allocated 10 vCPUs in an isolated way (take into account
  that threads number 1,13 and 0,12 are not available)

  If we boot a VM with only one vCPU in an isolated way, it will give an
  error (so expected behaviour):

  nova flavor-create iso auto 1024 10 1
  nova flavor-key iso set hw:cpu_policy=dedicated
  nova flavor-key iso set hw:cpu_thread_policy=isolate
  nova flavor-key iso set hw:numa_nodes=1
  nova boot testI --flavor iso --image cirros --nic net-id=$NET_ID

  If we boot a VM with one vCPU with m1.tiny flavor, it will be allowed:

  nova boot test --flavor 1 --image cirros --nic net-id=$NET_ID

  # nova list
  +--------------------------------------+----------+--------+------------+-------------+-----------------+
  | ID                                   | Name     | Status | Task State | Power State | Networks        |
  +--------------------------------------+----------+--------+------------+-------------+-----------------+
  | a669fbca-0607-44b5-b167-da49edf0276b | test     | ACTIVE | -          | Running     | om=192.168.1.17 |
  | 783a013c-78b8-4b66-89d9-eaab4e4d0ade | testI    | ERROR  | -          | NOSTATE     |                 |
  | 66e4a64d-b55e-4b37-94b9-c7330118467e | testIso1 | ACTIVE | -          | Running     | om=192.168.1.15 |
  | cd5e51b3-de30-4765-b407-a707385cb45c | testIso2 | ACTIVE | -          | Running     | om=192.168.1.16 |
  +--------------------------------------+----------+--------+------------+-------------+-----------------+

  # virsh vcpupin 4
  VCPU: CPU Affinity
  ----------------------------------
     0: 2-11,14-23

  So vCPUs are not properly isolated

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1578150/+subscriptions