yahoo-eng-team team mailing list archive

[Ricardo Noriega <rnoriega@xxxxxxxxxx>]

Public bug reported:

Description
===========
It seems that the main failure happens due to the incorrect NUMA filtering in the pci allocation mechanism. The allocation is being done according to the instance NUMA topology, however, this is not always correct. Specifically in the case when a user selects hw:numa_nodes=1, which would mean that VM will take resources from just one numa node and not from a specific one.


Steps to reproduce
==================

Create nova flavor with NUMA awareness, CPU pinning, Huge pages, etc:

#  nova flavor-create prefer_pin_1 auto 2048 20 1
#  nova flavor-key prefer_pin_1 set  hw:numa_nodes=1
#  nova flavor-key prefer_pin_1 set  hw:mem_page_size=1048576
#  nova flavor-key prefer_pin_1 set hw:numa_mempolicy=strict
#  nova flavor-key prefer_pin_1 set hw:cpu_policy=dedicated
#  nova flavor-key prefer_pin_1 set hw:cpu_thread_policy=prefer

Then instantiate VMs with direct-physical neutron ports:

neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf1
nova boot pf1 --flavor prefer_pin_1 --image centos_udev --nic port-id=a0fe88f6-07cc-4c70-b702-1915e36ed728
neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf2
nova boot pf2 --flavor prefer_pin_1 --image centos_udev --nic port-id=b96de3ec-ef94-428b-96bc-dc46623a2427

Third VM instantiation failed. Our environment has got 4 NICs configured
to be allocated. However, with a regular flavor (m1.normal), the
instantiation works:

neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf3
nova boot pf3 --flavor 2 --image centos_udev --nic port-id=52caacfe-0324-42bd-84ad-9a54d80e8fbe
neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf4
nova boot pf4 --flavor 2 --image centos_udev --nic port-id=7335a9a6-82d0-4595-bb88-754678db56ef


Expected result
===============

PCI passthrough (PFs and VFs) should work in an environment with
NUMATopologyFilter enable


Actual result
=============

Checking availability of NICs with NUMATopologyFilter is not working.


Environment
===========

1 controller + 1 compute.

OpenStack Mitaka

Logs & Configs
==============

See attachment

** Affects: nova
     Importance: Undecided
         Status: New

** Attachment added: "sosreport-nfv100.hi.inet-20160609134718.tar.xz"
   https://bugs.launchpad.net/bugs/1590746/+attachment/4680374/+files/sosreport-nfv100.hi.inet-20160609134718.tar.xz

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1590746

Title:
  SRIOV PF/VF allocation fails with NUMA aware flavor

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========
  It seems that the main failure happens due to the incorrect NUMA filtering in the pci allocation mechanism. The allocation is being done according to the instance NUMA topology, however, this is not always correct. Specifically in the case when a user selects hw:numa_nodes=1, which would mean that VM will take resources from just one numa node and not from a specific one.

  
  Steps to reproduce
  ==================

  Create nova flavor with NUMA awareness, CPU pinning, Huge pages, etc:

  #  nova flavor-create prefer_pin_1 auto 2048 20 1
  #  nova flavor-key prefer_pin_1 set  hw:numa_nodes=1
  #  nova flavor-key prefer_pin_1 set  hw:mem_page_size=1048576
  #  nova flavor-key prefer_pin_1 set hw:numa_mempolicy=strict
  #  nova flavor-key prefer_pin_1 set hw:cpu_policy=dedicated
  #  nova flavor-key prefer_pin_1 set hw:cpu_thread_policy=prefer

  Then instantiate VMs with direct-physical neutron ports:

  neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf1
  nova boot pf1 --flavor prefer_pin_1 --image centos_udev --nic port-id=a0fe88f6-07cc-4c70-b702-1915e36ed728
  neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf2
  nova boot pf2 --flavor prefer_pin_1 --image centos_udev --nic port-id=b96de3ec-ef94-428b-96bc-dc46623a2427

  Third VM instantiation failed. Our environment has got 4 NICs
  configured to be allocated. However, with a regular flavor
  (m1.normal), the instantiation works:

  neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf3
  nova boot pf3 --flavor 2 --image centos_udev --nic port-id=52caacfe-0324-42bd-84ad-9a54d80e8fbe
  neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf4
  nova boot pf4 --flavor 2 --image centos_udev --nic port-id=7335a9a6-82d0-4595-bb88-754678db56ef

  
  Expected result
  ===============

  PCI passthrough (PFs and VFs) should work in an environment with
  NUMATopologyFilter enable

  
  Actual result
  =============

  Checking availability of NICs with NUMATopologyFilter is not working.

  
  Environment
  ===========

  1 controller + 1 compute.

  OpenStack Mitaka

  Logs & Configs
  ==============

  See attachment

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1590746/+subscriptions