yahoo-eng-team team mailing list archive

[Daniel Alvarez <1723472@xxxxxxxxxxxxxxxxxx>]

We have seen that the MAC address of the FIP changes to the qf interface of a different controller.
However, the environment was running openstack-neutron-11.0.0-1.el7.noarch.

After upgrading to openstack-neutron-11.0.1-1.el7.noarch, this bug no longer occurs.
Marking it as invalid.

** Changed in: neutron
       Status: Confirmed => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1723472

Title:
  [DVR] Lowering the MTU breaks FIP traffic

Status in neutron:
  Invalid

Bug description:
  In a DVR environment, when lowering the MTU of a network, traffic
  going to an instance through a floating IP is broken.

  Description:

  * Ping traffic to a VM through its FIP works.
  * Change the MTU of its network through "neutron net-update <network> --mtu 1440".
  * Ping to the same FIP doesn't work.

  After a long debugging session with Anil Venkata, we've found that
  packets reach br-ex and then they hit this OF rule with normal action:

   cookie=0x1f847e4bf0de0aea, duration=70306.532s, table=3,
  n_packets=1579251, n_bytes=796614220, idle_age=0, hard_age=65534,
  priority=1 actions=NORMAL

  
  We would expect this rule to switch the packet to br-int so that it can be forwarded to the fip namespace (ie. with dst MAC address set to the floating ip gw (owner=network:floatingip_agent_gateway):

  $ sudo ovs-vsctl list interface

  _uuid               : 1f2b6e86-d303-42f4-9467-5dab78fc7199
  admin_state         : down
  bfd                 : {}
  bfd_status          : {}
  cfm_fault           : []
  cfm_fault_status    : []
  cfm_flap_count      : []
  cfm_health          : []
  cfm_mpid            : []
  cfm_remote_mpids    : []
  cfm_remote_opstate  : []
  duplex              : []
  error               : []
  external_ids        : {attached-mac="fa:16:3e:9d:0c:4f", iface-id="8ec34826-b1a6-48ce-9c39-2fd3e8167eb4", iface-status=active}
  name                : "fg-8ec34826-b1"


  [heat-admin@overcloud-novacompute-0 ~]$ sudo ovs-appctl fdb/show br-ex                                                                                                                        
   port  VLAN  MAC                Age
   [...]
      7    10  fa:16:3e:9d:0c:4f    0

  
  $ sudo ovs-ofctl show br-ex | grep "7("
   7(phy-br-ex): addr:36:63:93:fc:af:e2

  
  And from there, to the fip namespace which would route the packet to the qrouter namespace, etc.

  However, when we change the MTU through the following command:

  "neutron net-update <network> --mtu 1440"

  We see that, after a few seconds, the MAC address of the FIP changes
  so when traffic arrives br-ex and NORMAL action is performed, it will
  not be output to br-int through the patch-port but instead, through
  eth1 and traffic won't work anymore.

  [heat-admin@overcloud-novacompute-0 ~]$ arp -n | grep ".113"
  10.0.0.113               ether   fa:16:3e:9d:0c:4f   C                     vlan10

  neutron port-set xxxxx --mtu 1440

  $ arp -n | grep ".113"
  10.0.0.113               ether   fa:16:3e:20:f9:85   C                     vlan10

  
  When setting the MAC address manually, ping starts working again:

  $ arp -s 10.0.0.113 fa:16:3e:9d:0c:4f
  $ ping 10.0.0.113
  PING 10.0.0.113 (10.0.0.113) 56(84) bytes of data.
  64 bytes from 10.0.0.113: icmp_seq=1 ttl=62 time=1.17 ms
  64 bytes from 10.0.0.113: icmp_seq=2 ttl=62 time=0.561 ms

  
  Additional notes:

  When we set the MAC address manually and traffic gets working back
  again, lowering the MTU doesn't change the MAC address (we can't see
  any gARP's coming through).

  When we delete the ARP entry for the FIP and try to ping the FIP, the
  wrong MAC address is set.

  [heat-admin@overcloud-novacompute-0 ~]$ sudo arp -d 10.0.0.113

  [heat-admin@overcloud-novacompute-0 ~]$ ping 10.0.0.113 -c 2
  PING 10.0.0.113 (10.0.0.113) 56(84) bytes of data.

  --- 10.0.0.113 ping statistics ---
  2 packets transmitted, 0 received, 100% packet loss, time 999ms

  [heat-admin@overcloud-novacompute-0 ~]$ arp -n | grep ".113"
  10.0.0.113               ether   fa:16:3e:20:f9:85   C                     vlan10

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1723472/+subscriptions