yahoo-eng-team team mailing list archive

[OpenStack Infra <1533876@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/271444
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=adf7ba61dd73fe4bfffa20295be9a4b1006a1fe6
Submitter: Jenkins
Branch:    master

commit adf7ba61dd73fe4bfffa20295be9a4b1006a1fe6
Author: Sean Mooney <sean.k.mooney@xxxxxxxxx>
Date:   Fri Jan 22 17:00:36 2016 +0000

    stop setting mtu when plugging vhost-user ports
    
    vhost-user is a userspace protocol to establish connectivity
    between a virto-net frontend typically qemu and a
    userspace virtio backend such as ovs with dpdk.
    
    vhost-user interfaces exist only in userspace from the host perspective
    and are not represented in the linux networking stack as kernel netdevs.
    As a result attempting to set the mtu on a vhost-user interface
    using ifconfig or ip link will fail with a device not found error.
    
    - this change removes a call to _set_device_mtu when plugging
      vhost-user interfaces.
    - this change prevents the device not found error from occurring
      which stopped vms booting with vhost-user interfaces
      due to an uncaught exception resulting in a failure to set the
      interface type in ovs.
    - this change make creating vhost-user interface
      an atomic action.
    
    This latent bug is only triggered when the mtu value is set to a
    value other than 0 which was the default proir to mitaka.
    
    Change-Id: I2e17723d5052d57cd1557bd8a173c06ea0dcb2d4
    Closes-Bug: #1533876


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1533876

Title:
  plug_vhostuser may fail due to device not found error when setting mtu

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) kilo series:
  In Progress
Status in OpenStack Compute (nova) liberty series:
  In Progress
Status in OpenStack Compute (nova) mitaka series:
  In Progress

Bug description:
  Setting the mtu of a vhost-user port with the ip command will cause vms to fail
  to boot with a device not found error as vhost-user prots are not represented as
  kernel netdevs.

  this bug is present in stable/kilo, stable/liberty and master and i would like to ask that it be back ported if accepted
  and fixed in master.

  when using vhost-user with ovs-dpdk the vhost-user port is plugged
  into ovs by nova using a non atomic call to
  linux_net.create_ovs_vif_port to add an ovs port followed by a second
  call to linux_net.ovs_set_vhostuser_port_type to update the port type.

  https://github.com/openstack/nova/blob/1bf6a8760f0ef226dba927b62d4354e248b984de/nova/virt/libvirt/vif.py#L652-L655

  the reuse of the create_ovs_vif_port has an untended concequece of introducing an error where
  the ip tool is invoked to try and set the mtu on the userspace vhost-user interface which dose not exist
  as a kernel netdev.
  https://github.com/openstack/nova/blob/1bf6a8760f0ef226dba927b62d4354e248b984de/nova/network/linux_net.py#L1379

  this results in the in the call to set_device_mtu throwing an exception as the ip comand exits with code 1
  https://github.com/openstack/nova/blob/1bf6a8760f0ef226dba927b62d4354e248b984de/nova/network/linux_net.py#L1340-L1342

  as a result the second function call to ovs_set_vhostuser_port_type
  is never maid and the vm fails to boot.

  to resolve this issue i would like to introduce a new function to inux_net.py
  create_ovs_vhostuser_port  which will create the vhostuser port as an atomic action
  and will not set the mtu similar to the impentation in the os-vif vhost-user driver

  https://github.com/jaypipes/vif_plug_vhostuser/blob/8ac30ce32b3e0bae5d2d8f1edc9d64ac2871608e/vif_plug_vhostuser/linux_net.py#L34-L46

  an alternitive solution would be to add "1" to the retrun code check here https://github.com/openstack/nova/blob/master/nova/network/linux_net.py#L1339  or catch the exception here https://github.com/openstack/nova/blob/1bf6a8760f0ef226dba927b62d4354e248b984de/nova/virt/libvirt/vif.py#L652
   however neither solve the underlying cause.

  this was observed with kilo openstack  on ubuntu 14.04 with ovs-dpdk
  deployed with puppet/fule.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1533876/+subscriptions