yahoo-eng-team team mailing list archive

[piotrrr <1821299@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Associating Floating IP with a port can result in duplicate Floating
IPs, due to the original FIP not being removed from the SNAT namespace.
This is likely specific to using DVR.

We're creating a Heat stack containing, among other things, a Floating
IP and a Port.

  head_a_floating_ip:
    properties:
      floating_network_id: cf0a6df9-b533-457b-8cd7-0336f0649213
      port_id:
        get_resource: head_a_external_port
    type: OS::Neutron::FloatingIP
  head_a_external_port:
    properties:
      network_id:
        get_resource: external_net
      port_security_enabled: true
      replacement_policy: AUTO
      security_groups:
      - get_resource: head_sec_group
    type: OS::Neutron::Port

During this initial stack creation, we are not creating any VMs. So, the
port is not attached to any device.

It looks like because of those two lines in floating-ip definition
  port_id:
        get_resource: head_a_external_port
after the initial stack creation the Floating IP gets allocated in a SNAT namespace of one of the hypervisors, and starts to respond to ARP requests.

However, as soon as we update this stack, adding a VM, and making the
above mentioned port part of a VM, something weird happens. As expected,
Neutron then allocates that FIP on the hypervisor hosting the VM (as
expected, we're running DVR), however Neutron fails to remove the FIP it
had created initially in the SNAT namespace, after the initial stack
creation.

This results in FIP being present on two different hypervisors, causing
duplicate ARP replies (one MAC being in the SNAT namespace, the other in
the floating ip namespace), and obvious connectivity issues.

Note that the issues does not appear if the initial FIP happens to land
in the SNAT namespace of the same hypervisors which will later (after
stack update) also host the VM.

Simple, confirmed, workaround is to NOT include those two lines during the initial heat stack creation, and only include them in the stack update during which we add the VM.
  port_id:
        get_resource: head_a_external_port
Not including those lines initially in the stack results in Neutron not allocating the FIP anywhere.

Environment: Neutron Pike (11.0.5), with DVR, OVS, VLAN-based isolation.

** Affects: neutron
     Importance: Undecided
         Status: New

** Description changed:

  Associating Floating IP with a port can result in duplicate Floating
  IPs, due to the original FIP not being removed from the SNAT namespace.
  This is likely specific to using DVR.
  
+ We're creating a Heat stack containing, among other things, a Floating
+ IP and a Port.
  
- We're creating a Heat stack containing, among other things, a Floating IP and a Port.
- 
-   head_a_floating_ip:
-     properties:
-       floating_network_id: cf0a6df9-b533-457b-8cd7-0336f0649213
-       port_id:
-         get_resource: head_a_external_port
-     type: OS::Neutron::FloatingIP
-   head_a_external_port:
-     properties:
-       network_id:
-         get_resource: external_net
-       port_security_enabled: true
-       replacement_policy: AUTO
-       security_groups:
-       - get_resource: head_sec_group
-     type: OS::Neutron::Port
+   head_a_floating_ip:
+     properties:
+       floating_network_id: cf0a6df9-b533-457b-8cd7-0336f0649213
+       port_id:
+         get_resource: head_a_external_port
+     type: OS::Neutron::FloatingIP
+   head_a_external_port:
+     properties:
+       network_id:
+         get_resource: external_net
+       port_security_enabled: true
+       replacement_policy: AUTO
+       security_groups:
+       - get_resource: head_sec_group
+     type: OS::Neutron::Port
  
  During this initial stack creation, we are not creating any VMs. So, the
  port is not attached to any device.
  
  It looks like because of those two lines in floating-ip definition
-   port_id:
-         get_resource: head_a_external_port
+   port_id:
+         get_resource: head_a_external_port
  after the initial stack creation the Floating IP gets allocated in a SNAT namespace of one of the hypervisors, and starts to respond to ARP requests.
  
  However, as soon as we update this stack, adding a VM, and making the
  above mentioned port part of a VM, something weird happens. As expected,
  Neutron then allocates that FIP on the hypervisor hosting the VM (as
  expected, we're running DVR), however Neutron fails to remove the FIP it
  had created initially in the SNAT namespace, after the initial stack
  creation.
  
  This results in FIP being present on two different hypervisors, causing
  duplicate ARP replies (one MAC being in the SNAT namespace, the other in
  the floating ip namespace), and obvious connectivity issues.
  
  Note that the issues does not appear if the initial FIP happens to land
  in the SNAT namespace of the same hypervisors which will later (after
  stack update) also host the VM.
  
  Simple, confirmed, workaround is to NOT include those two lines during the initial heat stack creation, and only include them in the stack update during which we add the VM.
-   port_id:
-         get_resource: head_a_external_port
+   port_id:
+         get_resource: head_a_external_port
  Not including those lines initially in the stack results in Neutron not allocating the FIP anywhere.
+ 
+ Environment: Neutron Pike (11.0.5), with DVR, OVS, VLAN-based isolation.

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1821299

Title:
  Associating Floating IP with a port can result in duplicate Floating
  IPs, due to the original FIP not being removed from the SNAT
  namespace.

Status in neutron:
  New

Bug description:
  Associating Floating IP with a port can result in duplicate Floating
  IPs, due to the original FIP not being removed from the SNAT
  namespace. This is likely specific to using DVR.

  We're creating a Heat stack containing, among other things, a Floating
  IP and a Port.

    head_a_floating_ip:
      properties:
        floating_network_id: cf0a6df9-b533-457b-8cd7-0336f0649213
        port_id:
          get_resource: head_a_external_port
      type: OS::Neutron::FloatingIP
    head_a_external_port:
      properties:
        network_id:
          get_resource: external_net
        port_security_enabled: true
        replacement_policy: AUTO
        security_groups:
        - get_resource: head_sec_group
      type: OS::Neutron::Port

  During this initial stack creation, we are not creating any VMs. So,
  the port is not attached to any device.

  It looks like because of those two lines in floating-ip definition
    port_id:
          get_resource: head_a_external_port
  after the initial stack creation the Floating IP gets allocated in a SNAT namespace of one of the hypervisors, and starts to respond to ARP requests.

  However, as soon as we update this stack, adding a VM, and making the
  above mentioned port part of a VM, something weird happens. As
  expected, Neutron then allocates that FIP on the hypervisor hosting
  the VM (as expected, we're running DVR), however Neutron fails to
  remove the FIP it had created initially in the SNAT namespace, after
  the initial stack creation.

  This results in FIP being present on two different hypervisors,
  causing duplicate ARP replies (one MAC being in the SNAT namespace,
  the other in the floating ip namespace), and obvious connectivity
  issues.

  Note that the issues does not appear if the initial FIP happens to
  land in the SNAT namespace of the same hypervisors which will later
  (after stack update) also host the VM.

  Simple, confirmed, workaround is to NOT include those two lines during the initial heat stack creation, and only include them in the stack update during which we add the VM.
    port_id:
          get_resource: head_a_external_port
  Not including those lines initially in the stack results in Neutron not allocating the FIP anywhere.

  Environment: Neutron Pike (11.0.5), with DVR, OVS, VLAN-based
  isolation.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1821299/+subscriptions