← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #86617

[Bug 1936341] [NEW] Failure to get representor ports on NICs that link representor netdevs to PF PCI device

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

With os-vif vif_plug_ovs/linux_net.py from master and mlx5_core driver
with this commit [0] Nova will fail when attempting to plug ConnectX-5
representor ports. A recent change to OS-VIF [1] appears to have made an
attempt at fixing this, but AFAICT the fix is incomplete and does not
work in practice.

There is also a implementation merged to libvirt [2] which we might
glean correct details from.

It looks like there is code both in Nova PCI utils and OS-VIF that needs
to be updated to deal with this.

0: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=123f0f53dd64b67e34142485fe866a8a581f12f1
1: https://review.opendev.org/q/I49f6ae3f0e6bfbf555c8284bfd70371ce90da0c7
2: https://github.com/libvirt/libvirt/commit/5b1c525b1f3608156884aed0dc5e925306c1e260

A instrumented log excerpt:
2021-07-15 09:51:58.981 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_13" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.982 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_16" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.982 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="veth8e2d6f41" address_str="None" pf="False"
2021-07-15 09:51:58.982 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_5" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.983 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2924" address_str="None" pf="False"
2021-07-15 09:51:58.983 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_31" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.983 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="tap9e7c516c-b0" address_str="None" pf="False"
2021-07-15 09:51:58.984 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_21" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.984 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_0" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.984 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_24" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.984 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_11" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.985 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.985 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_14" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.985 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_3" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.985 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2922" address_str="None" pf="False"
2021-07-15 09:51:58.986 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_9" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.986 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_22" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.986 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="lo" address_str="None" pf="False"
2021-07-15 09:51:58.987 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="vethffd8a2a7" address_str="None" pf="False"
2021-07-15 09:51:58.987 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="eno4" address_str="0000:05:00.1" pf="False"
2021-07-15 09:51:58.987 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_12" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.988 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_1" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.988 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_28" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.988 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_7" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.988 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_30" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.989 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="lxdbr0" address_str="None" pf="False"
2021-07-15 09:51:58.989 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_18" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.989 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_20" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.990 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="eno2" address_str="0000:01:00.1" pf="True"
2021-07-15 09:51:58.990 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_10" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.990 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond1.2925" address_str="None" pf="False"
2021-07-15 09:51:58.990 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_26" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.991 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_5" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.991 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0" address_str="None" pf="False"
2021-07-15 09:51:58.991 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_29" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.991 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_16" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.992 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_19" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.992 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp130s0f0" address_str="0000:82:00.0" pf="True"
2021-07-15 09:51:58.992 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_8" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.993 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2927" address_str="None" pf="False"
2021-07-15 09:51:58.993 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="vethc8b8e217" address_str="None" pf="False"
2021-07-15 09:51:58.994 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_24" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.994 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_3" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.994 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="vethb03153dd" address_str="None" pf="False"
2021-07-15 09:51:58.995 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_27" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.995 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2935" address_str="None" pf="False"
2021-07-15 09:51:58.995 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_14" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.996 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_17" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.996 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_6" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.996 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bonding_masters" address_str="None" pf="False"
2021-07-15 09:51:58.996 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_22" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.997 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_1" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.997 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="veth9849aea4" address_str="None" pf="False"
2021-07-15 09:51:58.998 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_25" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.998 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2933" address_str="None" pf="False"
2021-07-15 09:51:58.998 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_12" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:58.999 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="genev_sys_6081" address_str="None" pf="False"
2021-07-15 09:51:58.999 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_15" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:58.999 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_4" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:59.000 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2923" address_str="None" pf="False"
2021-07-15 09:51:59.000 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_30" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.000 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="br-bond0" address_str="None" pf="False"
2021-07-15 09:51:59.000 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="vethceb69347" address_str="None" pf="False"
2021-07-15 09:51:59.001 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_20" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.001 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_23" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:59.001 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_10" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.002 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.002 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_13" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:59.002 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="ovs-system" address_str="None" pf="False"
2021-07-15 09:51:59.003 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_2" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:59.003 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_29" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.003 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_8" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.003 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_31" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:59.004 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_19" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.004 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_21" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:59.004 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="br-ex" address_str="None" pf="False"
2021-07-15 09:51:59.005 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="eno3" address_str="0000:05:00.0" pf="False"
2021-07-15 09:51:59.005 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_11" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:59.005 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_0" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:59.006 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_27" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.006 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_6" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.006 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond1" address_str="None" pf="False"
2021-07-15 09:51:59.006 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_17" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.007 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp130s0f1" address_str="0000:82:00.1" pf="True"
2021-07-15 09:51:59.007 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_9" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:59.007 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="eno1" address_str="0000:01:00.0" pf="True"
2021-07-15 09:51:59.008 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_25" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.008 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_4" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.008 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_28" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:59.008 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2936" address_str="None" pf="False"
2021-07-15 09:51:59.009 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_15" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.009 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_18" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:59.009 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_7" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:59.009 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2926" address_str="None" pf="False"
2021-07-15 09:51:59.010 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="br-int" address_str="None" pf="False"
2021-07-15 09:51:59.010 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_23" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.010 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_2" address_str="0000:03:00.0" pf="True"
2021-07-15 09:51:59.011 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_26" address_str="0000:03:00.1" pf="True"
2021-07-15 09:51:59.011 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2934" address_str="None" pf="False"
2021-07-15 09:51:59.011 1561931 ERROR os_vif [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] Failed to plug vif VIFHostDevice(active=True,address=fa:16:3e:e1:b1:f6,dev_address=0000:03:04.1,dev_type='ethernet',has_traffic_filtering=True,id=a0a41556-c0a3-4725-bcd9-b21644329e2c,network=Network(9e7c516c-b00b-4f7c-89ec-0eaa0519b046),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True): vif_plug_ovs.exception.RepresentorNotFound: Failed getting representor port for PF enp3s0f0 with 31
2021-07-15 09:51:59.011 1561931 ERROR os_vif Traceback (most recent call last):
2021-07-15 09:51:59.011 1561931 ERROR os_vif   File "/usr/lib/python3/dist-packages/os_vif/__init__.py", line 77, in plug
2021-07-15 09:51:59.011 1561931 ERROR os_vif     plugin.plug(vif, instance_info)
2021-07-15 09:51:59.011 1561931 ERROR os_vif   File "/usr/lib/python3/dist-packages/vif_plug_ovs/ovs.py", line 299, in plug
2021-07-15 09:51:59.011 1561931 ERROR os_vif     self._plug_vf(vif, instance_info)
2021-07-15 09:51:59.011 1561931 ERROR os_vif   File "/usr/lib/python3/dist-packages/vif_plug_ovs/ovs.py", line 265, in _plug_vf
2021-07-15 09:51:59.011 1561931 ERROR os_vif     representor = linux_net.get_representor_port(pf_ifname, vf_num)
2021-07-15 09:51:59.011 1561931 ERROR os_vif   File "/usr/lib/python3/dist-packages/vif_plug_ovs/linux_net.py", line 312, in get_representor_port
2021-07-15 09:51:59.011 1561931 ERROR os_vif     raise exception.RepresentorNotFound(ifname=pf_ifname, vf_num=vf_num)
2021-07-15 09:51:59.011 1561931 ERROR os_vif vif_plug_ovs.exception.RepresentorNotFound: Failed getting representor port for PF enp3s0f0 with 31
2021-07-15 09:51:59.011 1561931 ERROR os_vif 
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] Error starting thread.: nova.exception.InternalError: Failure running os_vif plugin plug method: Failed to plug VIF VIFHostDevice(active=True,address=fa:16:3e:e1:b1:f6,dev_address=0000:03:04.1,dev_type='ethernet',has_traffic_filtering=True,id=a0a41556-c0a3-4725-bcd9-b21644329e2c,network=Network(9e7c516c-b00b-4f7c-89ec-0eaa0519b046),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True). Got error: Failed getting representor port for PF enp3s0f0 with 31
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service Traceback (most recent call last):
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/os_vif/__init__.py", line 77, in plug
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     plugin.plug(vif, instance_info)
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/vif_plug_ovs/ovs.py", line 299, in plug
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     self._plug_vf(vif, instance_info)
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/vif_plug_ovs/ovs.py", line 265, in _plug_vf
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     representor = linux_net.get_representor_port(pf_ifname, vf_num)
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/vif_plug_ovs/linux_net.py", line 312, in get_representor_port
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     raise exception.RepresentorNotFound(ifname=pf_ifname, vf_num=vf_num)
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service vif_plug_ovs.exception.RepresentorNotFound: Failed getting representor port for PF enp3s0f0 with 31
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service 
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service During handling of the above exception, another exception occurred:
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service 
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service Traceback (most recent call last):
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/vif.py", line 738, in _plug_os_vif
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     os_vif.plug(vif, instance_info)
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/os_vif/__init__.py", line 82, in plug
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     raise os_vif.exception.PlugException(vif=vif, err=err)
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service os_vif.exception.PlugException: Failed to plug VIF VIFHostDevice(active=True,address=fa:16:3e:e1:b1:f6,dev_address=0000:03:04.1,dev_type='ethernet',has_traffic_filtering=True,id=a0a41556-c0a3-4725-bcd9-b21644329e2c,network=Network(9e7c516c-b00b-4f7c-89ec-0eaa0519b046),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True). Got error: Failed getting representor port for PF enp3s0f0 with 31
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service 
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service During handling of the above exception, another exception occurred:
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service 
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service Traceback (most recent call last):
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/oslo_service/service.py", line 810, in run_service
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     service.start()
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/service.py", line 173, in start
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     self.manager.init_host()
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 1433, in init_host
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     self._init_instance(context, instance)
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 1128, in _init_instance
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     self.driver.plug_vifs(instance, net_info)
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 1152, in plug_vifs
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     self.vif_driver.plug(instance, vif)
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/vif.py", line 762, in plug
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     self._plug_os_vif(instance, vif_obj)
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/vif.py", line 742, in _plug_os_vif
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     raise exception.InternalError(msg)
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service nova.exception.InternalError: Failure running os_vif plugin plug method: Failed to plug VIF VIFHostDevice(active=True,address=fa:16:3e:e1:b1:f6,dev_address=0000:03:04.1,dev_type='ethernet',has_traffic_filtering=True,id=a0a41556-c0a3-4725-bcd9-b21644329e2c,network=Network(9e7c516c-b00b-4f7c-89ec-0eaa0519b046),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True). Got error: Failed getting representor port for PF enp3s0f0 with 31
2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service

** Affects: nova
     Importance: Undecided
         Status: New

** Affects: os-vif
     Importance: Undecided
         Status: New

** Also affects: nova
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1936341

Title:
  Failure to get representor ports on NICs that link representor netdevs
  to PF PCI device

Status in OpenStack Compute (nova):
  New
Status in os-vif:
  New

Bug description:
  With os-vif vif_plug_ovs/linux_net.py from master and mlx5_core driver
  with this commit [0] Nova will fail when attempting to plug ConnectX-5
  representor ports. A recent change to OS-VIF [1] appears to have made
  an attempt at fixing this, but AFAICT the fix is incomplete and does
  not work in practice.

  There is also a implementation merged to libvirt [2] which we might
  glean correct details from.

  It looks like there is code both in Nova PCI utils and OS-VIF that
  needs to be updated to deal with this.

  0: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=123f0f53dd64b67e34142485fe866a8a581f12f1
  1: https://review.opendev.org/q/I49f6ae3f0e6bfbf555c8284bfd70371ce90da0c7
  2: https://github.com/libvirt/libvirt/commit/5b1c525b1f3608156884aed0dc5e925306c1e260

  A instrumented log excerpt:
  2021-07-15 09:51:58.981 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_13" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.982 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_16" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.982 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="veth8e2d6f41" address_str="None" pf="False"
  2021-07-15 09:51:58.982 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_5" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.983 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2924" address_str="None" pf="False"
  2021-07-15 09:51:58.983 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_31" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.983 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="tap9e7c516c-b0" address_str="None" pf="False"
  2021-07-15 09:51:58.984 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_21" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.984 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_0" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.984 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_24" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.984 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_11" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.985 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.985 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_14" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.985 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_3" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.985 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2922" address_str="None" pf="False"
  2021-07-15 09:51:58.986 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_9" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.986 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_22" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.986 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="lo" address_str="None" pf="False"
  2021-07-15 09:51:58.987 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="vethffd8a2a7" address_str="None" pf="False"
  2021-07-15 09:51:58.987 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="eno4" address_str="0000:05:00.1" pf="False"
  2021-07-15 09:51:58.987 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_12" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.988 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_1" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.988 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_28" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.988 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_7" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.988 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_30" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.989 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="lxdbr0" address_str="None" pf="False"
  2021-07-15 09:51:58.989 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_18" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.989 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_20" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.990 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="eno2" address_str="0000:01:00.1" pf="True"
  2021-07-15 09:51:58.990 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_10" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.990 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond1.2925" address_str="None" pf="False"
  2021-07-15 09:51:58.990 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_26" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.991 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_5" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.991 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0" address_str="None" pf="False"
  2021-07-15 09:51:58.991 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_29" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.991 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_16" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.992 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_19" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.992 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp130s0f0" address_str="0000:82:00.0" pf="True"
  2021-07-15 09:51:58.992 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_8" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.993 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2927" address_str="None" pf="False"
  2021-07-15 09:51:58.993 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="vethc8b8e217" address_str="None" pf="False"
  2021-07-15 09:51:58.994 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_24" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.994 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_3" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.994 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="vethb03153dd" address_str="None" pf="False"
  2021-07-15 09:51:58.995 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_27" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.995 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2935" address_str="None" pf="False"
  2021-07-15 09:51:58.995 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_14" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.996 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_17" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.996 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_6" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.996 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bonding_masters" address_str="None" pf="False"
  2021-07-15 09:51:58.996 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_22" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.997 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_1" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.997 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="veth9849aea4" address_str="None" pf="False"
  2021-07-15 09:51:58.998 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_25" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.998 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2933" address_str="None" pf="False"
  2021-07-15 09:51:58.998 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_12" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:58.999 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="genev_sys_6081" address_str="None" pf="False"
  2021-07-15 09:51:58.999 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_15" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:58.999 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_4" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:59.000 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2923" address_str="None" pf="False"
  2021-07-15 09:51:59.000 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_30" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.000 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="br-bond0" address_str="None" pf="False"
  2021-07-15 09:51:59.000 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="vethceb69347" address_str="None" pf="False"
  2021-07-15 09:51:59.001 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_20" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.001 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_23" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:59.001 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_10" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.002 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.002 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_13" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:59.002 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="ovs-system" address_str="None" pf="False"
  2021-07-15 09:51:59.003 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_2" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:59.003 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_29" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.003 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_8" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.003 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_31" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:59.004 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_19" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.004 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_21" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:59.004 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="br-ex" address_str="None" pf="False"
  2021-07-15 09:51:59.005 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="eno3" address_str="0000:05:00.0" pf="False"
  2021-07-15 09:51:59.005 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_11" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:59.005 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_0" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:59.006 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_27" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.006 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_6" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.006 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond1" address_str="None" pf="False"
  2021-07-15 09:51:59.006 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_17" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.007 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp130s0f1" address_str="0000:82:00.1" pf="True"
  2021-07-15 09:51:59.007 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_9" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:59.007 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="eno1" address_str="0000:01:00.0" pf="True"
  2021-07-15 09:51:59.008 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_25" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.008 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_4" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.008 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_28" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:59.008 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2936" address_str="None" pf="False"
  2021-07-15 09:51:59.009 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_15" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.009 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_18" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:59.009 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_7" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:59.009 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2926" address_str="None" pf="False"
  2021-07-15 09:51:59.010 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="br-int" address_str="None" pf="False"
  2021-07-15 09:51:59.010 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_23" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.010 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f0_2" address_str="0000:03:00.0" pf="True"
  2021-07-15 09:51:59.011 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="enp3s0f1_26" address_str="0000:03:00.1" pf="True"
  2021-07-15 09:51:59.011 1561931 INFO vif_plug_ovs.linux_net [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] HELLO 0 device="bond0.2934" address_str="None" pf="False"
  2021-07-15 09:51:59.011 1561931 ERROR os_vif [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] Failed to plug vif VIFHostDevice(active=True,address=fa:16:3e:e1:b1:f6,dev_address=0000:03:04.1,dev_type='ethernet',has_traffic_filtering=True,id=a0a41556-c0a3-4725-bcd9-b21644329e2c,network=Network(9e7c516c-b00b-4f7c-89ec-0eaa0519b046),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True): vif_plug_ovs.exception.RepresentorNotFound: Failed getting representor port for PF enp3s0f0 with 31
  2021-07-15 09:51:59.011 1561931 ERROR os_vif Traceback (most recent call last):
  2021-07-15 09:51:59.011 1561931 ERROR os_vif   File "/usr/lib/python3/dist-packages/os_vif/__init__.py", line 77, in plug
  2021-07-15 09:51:59.011 1561931 ERROR os_vif     plugin.plug(vif, instance_info)
  2021-07-15 09:51:59.011 1561931 ERROR os_vif   File "/usr/lib/python3/dist-packages/vif_plug_ovs/ovs.py", line 299, in plug
  2021-07-15 09:51:59.011 1561931 ERROR os_vif     self._plug_vf(vif, instance_info)
  2021-07-15 09:51:59.011 1561931 ERROR os_vif   File "/usr/lib/python3/dist-packages/vif_plug_ovs/ovs.py", line 265, in _plug_vf
  2021-07-15 09:51:59.011 1561931 ERROR os_vif     representor = linux_net.get_representor_port(pf_ifname, vf_num)
  2021-07-15 09:51:59.011 1561931 ERROR os_vif   File "/usr/lib/python3/dist-packages/vif_plug_ovs/linux_net.py", line 312, in get_representor_port
  2021-07-15 09:51:59.011 1561931 ERROR os_vif     raise exception.RepresentorNotFound(ifname=pf_ifname, vf_num=vf_num)
  2021-07-15 09:51:59.011 1561931 ERROR os_vif vif_plug_ovs.exception.RepresentorNotFound: Failed getting representor port for PF enp3s0f0 with 31
  2021-07-15 09:51:59.011 1561931 ERROR os_vif 
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service [req-0fefa4d9-069e-41d5-8d4d-13b78a4a2475 - - - - -] Error starting thread.: nova.exception.InternalError: Failure running os_vif plugin plug method: Failed to plug VIF VIFHostDevice(active=True,address=fa:16:3e:e1:b1:f6,dev_address=0000:03:04.1,dev_type='ethernet',has_traffic_filtering=True,id=a0a41556-c0a3-4725-bcd9-b21644329e2c,network=Network(9e7c516c-b00b-4f7c-89ec-0eaa0519b046),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True). Got error: Failed getting representor port for PF enp3s0f0 with 31
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service Traceback (most recent call last):
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/os_vif/__init__.py", line 77, in plug
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     plugin.plug(vif, instance_info)
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/vif_plug_ovs/ovs.py", line 299, in plug
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     self._plug_vf(vif, instance_info)
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/vif_plug_ovs/ovs.py", line 265, in _plug_vf
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     representor = linux_net.get_representor_port(pf_ifname, vf_num)
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/vif_plug_ovs/linux_net.py", line 312, in get_representor_port
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     raise exception.RepresentorNotFound(ifname=pf_ifname, vf_num=vf_num)
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service vif_plug_ovs.exception.RepresentorNotFound: Failed getting representor port for PF enp3s0f0 with 31
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service 
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service During handling of the above exception, another exception occurred:
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service 
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service Traceback (most recent call last):
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/vif.py", line 738, in _plug_os_vif
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     os_vif.plug(vif, instance_info)
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/os_vif/__init__.py", line 82, in plug
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     raise os_vif.exception.PlugException(vif=vif, err=err)
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service os_vif.exception.PlugException: Failed to plug VIF VIFHostDevice(active=True,address=fa:16:3e:e1:b1:f6,dev_address=0000:03:04.1,dev_type='ethernet',has_traffic_filtering=True,id=a0a41556-c0a3-4725-bcd9-b21644329e2c,network=Network(9e7c516c-b00b-4f7c-89ec-0eaa0519b046),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True). Got error: Failed getting representor port for PF enp3s0f0 with 31
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service 
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service During handling of the above exception, another exception occurred:
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service 
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service Traceback (most recent call last):
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/oslo_service/service.py", line 810, in run_service
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     service.start()
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/service.py", line 173, in start
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     self.manager.init_host()
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 1433, in init_host
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     self._init_instance(context, instance)
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 1128, in _init_instance
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     self.driver.plug_vifs(instance, net_info)
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 1152, in plug_vifs
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     self.vif_driver.plug(instance, vif)
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/vif.py", line 762, in plug
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     self._plug_os_vif(instance, vif_obj)
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/vif.py", line 742, in _plug_os_vif
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service     raise exception.InternalError(msg)
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service nova.exception.InternalError: Failure running os_vif plugin plug method: Failed to plug VIF VIFHostDevice(active=True,address=fa:16:3e:e1:b1:f6,dev_address=0000:03:04.1,dev_type='ethernet',has_traffic_filtering=True,id=a0a41556-c0a3-4725-bcd9-b21644329e2c,network=Network(9e7c516c-b00b-4f7c-89ec-0eaa0519b046),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True). Got error: Failed getting representor port for PF enp3s0f0 with 31
  2021-07-15 09:51:59.100 1561931 ERROR oslo_service.service

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1936341/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next