yahoo-eng-team team mailing list archive

[frigo <1960405@xxxxxxxxxxxxxxxxxx>]

I can't reproduce. I think the problem was that the ovn controller was
not claiming ports as needed, due to my own development environment. I
can't know for sure, but to investigate such issue I would first look
into any

    Claiming virtual lport 
    Releasing lport

from the ovn-controller logs on the chassis, and try to correlate with actions running at that time in the neutron server. 
closing


** Changed in: neutron
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1960405

Title:
  snat is used instead of dnat_and_snat when L3GW resides on chassis

Status in neutron:
  Invalid

Bug description:
  I run RHOSP 16.2 (Train) with OVN and enable_distributed_floating_ip.
  On the same chassis I have a VM running and the L3GW port scheduled there, and a FIP is associated to the VM.
  I would expect the "dnat_and_snat" NAT to be used and traffic to egress with the FIP. However, as the L3GW is scheduled there too, I see the "snat" NAT is used instead.

  I think this is a bug (unless I'm wrong)
  - a VM having a FIP should use this FIP for egress traffic. External firewall expect it
  - the L3GW port is expected to move. If the port moves to a chassis where the traffic is already flowing using the FIP, the presence of the L3GW port should not disrupt the traffic.


  
  * Reproduction steps
  We assume 2 chassis, cpu34d and cpu35d.
  # Create a router, we make sure its port is scheduled on cpu35d:

  openstack router create router1 --availability-zone-hint a35
  openstack router set --external-gateway external1 --fixed-ip subnet=tenant_35 router1
  openstack port show 34ef841f-545f-4fab-9447-11bf18ae0e1a \
    -c binding_host_id -c device_owner -c fixed_ips
  +-----------------+------------------------------------------------------------------------------+
  | Field           | Value                                                                        |
  +-----------------+------------------------------------------------------------------------------+
  | binding_host_id | cpu35d                              
  | device_owner    | network:router_gateway                                                       |
  | fixed_ips       | ip_address='10.64.245.126', subnet_id='e955b866-324d-491f-888c-2760b713d3b0' |
  +-----------------+------------------------------------------------------------------------------+
  openstack router add subnet router1 mysub

  # We run a VM on cpu35d with floating IP 10.64.254.128 associated
  openstack server create myserver --key-name stack --security-group prodlike --network private \
   --image cirros --flavor m1.small --availability-zone nova:cpu35d
  openstack server add floating ip myserver 10.64.254.128
  ssh cirros@10.64.254.128
  ping external

  # We run another VM, on the other chassis:
  openstack server create myserver2 --key-name stack --security-group prodlike --network private \
   --image cirros --flavor m1.small --availability-zone nova:cpu34d
  openstack server add floating ip myserver2 10.64.254.135
  ssh cirros@10.64.254.135
  ping external

  We observe the egress traffic:

  
  * Expected output: what did you hope to see?
  from myserver: traffic coming from the fip:
  IP 10.64.254.128 > external

  from myserver2:
  IP 10.64.254.135 > external


  * Actual output:
  from myserver: traffic coming from the L3GW port:
  IP 10.64.245.126 > external

  from myserver2:
  IP 10.64.254.135 > external

  
  * Version:
    ** OpenStack version RHOSP 16.2 (Train)
    ** RHEL 8.4
    ** deployed with tripleo (ovn-2021-central-21.09.1-20.el8fdp.x86_64, python3-neutron-15.3.5-2.20210608154816.el8ost.4.noarch

  
  From OVN northdb:
  router 6a1c6c1d-e365-4684-96d6-9b06e4ad5862 (neutron-abf4070d-6134-4bf8-b398-a9e201b66b08) (aka router1)
      port lrp-34ef841f-545f-4fab-9447-11bf18ae0e1a
          mac: "fa:16:3e:22:3f:c0"
          networks: ["10.64.245.126/27"]
          gateway chassis: [1126ea9a-2860-4e5c-9ab5-ca1e8959edee]
      port lrp-e56e108f-d731-45e1-ba45-444219572859
          mac: "fa:16:3e:90:a7:3b"
          networks: ["192.168.200.1/27"]
      nat 8e72663f-2c9d-49fa-9749-223df298c646
          external ip: "10.64.254.128"
          logical ip: "192.168.200.21"
          type: "dnat_and_snat"
      nat b0d9f69a-c8d4-4413-8f00-1c6f0b9e643f
          external ip: "10.64.245.126"
          logical ip: "192.168.200.0/27"
          type: "snat"
      nat f5808a3a-f40c-4ccf-a9fe-b83209011555
          external ip: "10.64.254.135"
          logical ip: "192.168.200.27"
          type: "dnat_and_snat"

  From ovn-trace we read:

  ingress(dp="router1", inport="lrp-e56e10")
  ------------------------------------------
   0. lr_in_admission (northd.c:10285): eth.dst == fa:16:3e:90:a7:3b && inport == "lrp-e56e10", priority 50, uuid 3711664a
      xreg0[0..47] = fa:16:3e:90:a7:3b;
      next;
   1. lr_in_lookup_neighbor (northd.c:10365): 1, priority 0, uuid a7f32214
      reg9[2] = 1;
      next;
   2. lr_in_learn_neighbor (northd.c:10374): reg9[2] == 1, priority 100, uuid bb51e95e
      next;
  10. lr_in_ip_routing (northd.c:9179): ip4.dst == 0.0.0.0/0, priority 1, uuid 57ef0971
      ip.ttl--;
      reg8[0..15] = 0;
      reg0 = 10.64.245.97;
      reg1 = 10.64.245.126;
      eth.src = fa:16:3e:22:3f:c0;
      outport = "lrp-34ef84";
      flags.loopback = 1;
      next;
  11. lr_in_ip_routing_ecmp (northd.c:10670): reg8[0..15] == 0, priority 150, uuid 9b75d8f6
      next;
  12. lr_in_policy (northd.c:10795): 1, priority 0, uuid a68ffb22
      reg8[0..15] = 0;
      next;
  13. lr_in_policy_ecmp (northd.c:10797): reg8[0..15] == 0, priority 150, uuid c2a799d9
      next;
  14. lr_in_arp_resolve (northd.c:10831): ip4, priority 0, uuid 3dff7cbd
      get_arp(outport, reg0);
      /* MAC binding to 00:1c:73:00:00:11. */
      next;
  17. lr_in_gw_redirect (northd.c:12774): ip4.src == 192.168.200.21 && outport == "lrp-34ef84" && is_chassis_resident("464051"), priority 100, uuid f142bc57
      eth.src = fa:16:3e:c3:29:10;
      reg1 = 10.64.254.128;
      next;
  18. lr_in_arp_request (northd.c:11488): 1, priority 0, uuid 39a08290
      output;

  egress(dp="router1", inport="lrp-e56e10", outport="lrp-34ef84")
  ---------------------------------------------------------------
   0. lr_out_undnat (northd.c:12318): ip && ip4.src == 192.168.200.21 && outport == "lrp-34ef84", priority 100, uuid 8d276994
      eth.src = fa:16:3e:c3:29:10;
      ct_dnat;

  ct_dnat /* assuming no un-dnat entry, so no change */
  -----------------------------------------------------
   2. lr_out_snat (northd.c:12410): ip && ip4.src == 192.168.200.0/27 && outport == "lrp-34ef84" && is_chassis_resident("cr-lrp-34ef84"), priority 156, uuid 7a29e2cf
      ct_snat(10.64.245.126);

  ct_snat(ip4.src=10.64.245.126)
  ------------------------------
   4. lr_out_delivery (northd.c:11536): outport == "lrp-34ef84", priority 100, uuid d00dd976
      output;
      /* output to "lrp-34ef84", type "patch" */


  I think what is happening is is_chassis_resident("cr-lrp-34ef84") is
  running in the end and override the source IP with its own.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1960405/+subscriptions