yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #96434
[Bug 2124259] [NEW] [ovn] Direct routing (non NAT) does not work between VLAN and Geneve networks
Public bug reported:
Environment:
OpenStack Kolla-Ansible deployed 2025.1
Rocky Linux 9.6
OVN version: latest 24.03 from CentOS NFV SIG - ovn24.03-24.03.6-22.el9s.x86_64
OVS version: openvswitch3.3-3.3.4-115.el9s.x86_64
Distributed floating IP enabled
Running with a backport of Gateway_Chassis to HA_Chassis_Group patch
stack (https://review.opendev.org/c/openstack/neutron/+/947317/13)
Router with multiple Geneve and VLAN networks (plus a provider network
with gateway):
192.168.44.0/24 - VLAN type network
172.16.0.1/23 - Geneve type network
masked_internet_ip/28 - VLAN type provider network (FIP and external gateway)
ovn-nbctl show output for given router:
router 1c387a14-81d8-4277-88bb-9a8307599991 (neutron-a1970ca5-0e53-4c22-9199-6f919903335b) (aka external)
port lrp-c4ca7855-b4ba-45c0-b963-1cdb9bc6a423
mac: "fa:16:3e:6f:4e:b4"
networks: ["172.16.0.1/23"]
port lrp-7508355e-87ce-4ba4-b3e3-ebd56afeec32
mac: "fa:16:3e:a7:78:79"
networks: ["192.168.44.1/24"]
port lrp-3eb71d03-923e-4434-8502-791b2df8dc0c
mac: "fa:16:3e:19:25:a0"
networks: ["masked_internet_ip/28", "masked_internet_ipv6/127"]
# ovn-nbctl find Logical_Router_Port name=lrp-7508355e-87ce-4ba4-b3e3-ebd56afeec32
_uuid : ab91bd65-4830-4fcd-919a-f87aa03b06e5
enabled : []
external_ids : {"neutron:is_ext_gw"=False, "neutron:network_name"=neutron-c557a6bb-ad28-4479-972a-4e842f328d3a, "neutron:revision_number"="2", "neutron:router_name"="a1970ca5-0e53-4c22-9199-6f919903335b", "neutron:subnet_ids"="c4e30d02-b219-4c4c-85f5-7859ddc88193"}
gateway_chassis : []
ha_chassis_group : []
ipv6_prefix : []
ipv6_ra_configs : {}
mac : "fa:16:3e:a7:78:79"
name : lrp-7508355e-87ce-4ba4-b3e3-ebd56afeec32
networks : ["192.168.44.1/24"]
options : {reside-on-redirect-chassis="true"}
peer : []
status : {}
# ovn-nbctl find Logical_Router_Port name=lrp-c4ca7855-b4ba-45c0-b963-1cdb9bc6a423
_uuid : 86514488-f62a-4982-80b0-3e9e1177185a
enabled : []
external_ids : {"neutron:is_ext_gw"=False, "neutron:network_name"=neutron-e2b9e59f-43da-4e1c-b558-dc9da4c0d738, "neutron:revision_number"="2", "neutron:router_name"="a1970ca5-0e53-4c22-9199-6f919903335b", "neutron:subnet_ids"="724f792d-183f-406b-b207-02050126813f"}
gateway_chassis : []
ha_chassis_group : []
ipv6_prefix : []
ipv6_ra_configs : {}
mac : "fa:16:3e:6f:4e:b4"
name : lrp-c4ca7855-b4ba-45c0-b963-1cdb9bc6a423
networks : ["172.16.0.1/23"]
options : {}
peer : []
status : {}
# ovn-nbctl find Logical_Router_Port name=lrp-3eb71d03-923e-4434-8502-791b2df8dc0c
_uuid : 79e57ba7-b7d3-4174-a170-9341cc3210eb
enabled : []
external_ids : {"neutron:is_ext_gw"=True, "neutron:network_name"=neutron-80e70560-297d-498e-9acc-51e7822d09a8, "neutron:revision_number"="492", "neutron:router_name"="a1970ca5-0e53-4c22-9199-6f919903335b", "neutron:subnet_ids"="1674bc43-b481-4e13-ad17-c1b80144a282 616f7216-c329-419b-bf39-791b58babd74"}
gateway_chassis : []
ha_chassis_group : 9c8a379b-6a3d-4a1b-84ab-c1d6bd9096c2
ipv6_prefix : []
ipv6_ra_configs : {}
mac : "fa:16:3e:19:25:a0"
name : lrp-3eb71d03-923e-4434-8502-791b2df8dc0c
networks : ["masked_internet_ip/28", "masked_internet_ipv6/127"]
options : {reside-on-redirect-chassis="true"}
peer : []
status : {hosting-chassis=controller03}
Running ping from a VM with address 172.16.0.149 (Geneve network) to a
VM with address 192.168.44.228 gives ICMP replies in the first ,,pinging
session'':
$ ping 192.168.44.228
PING 192.168.44.228 (192.168.44.228) 56(84) bytes of data.
64 bytes from 192.168.44.228: icmp_seq=1 ttl=63 time=3.02 ms
64 bytes from 192.168.44.228: icmp_seq=2 ttl=63 time=1.40 ms
^C
--- 192.168.44.228 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 1.396/2.208/3.021/0.812 ms
And then nothing comes back in the second session:
$ ping 192.168.44.228
PING 192.168.44.228 (192.168.44.228) 56(84) bytes of data.
^C
--- 192.168.44.228 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2038ms
the ICMP reply packets are visible in tcpdump on the controller (network
gateway) - but are not reaching the VM (172.16.0.149)
After some minutes of no traffic between these hosts - the first
,,pinging session'' gets some ICMP replies, but after ctrl+c and running
it again - no replies
Removing reside-on-redirect-chassis=true from the 192.168.44.1 LRP
brings the traffic between these VMs to normal - but SNAT from
192.168.44.0/24 to the internet stops working.
** Affects: neutron
Importance: Undecided
Status: New
** Description changed:
Environment:
OpenStack Kolla-Ansible deployed 2025.1
+ Rocky Linux 9.6
+ OVN version: latest 24.03 from CentOS NFV SIG - ovn24.03-24.03.6-22.el9s.x86_64
+ OVS version: openvswitch3.3-3.3.4-115.el9s.x86_64
Distributed floating IP enabled
Running with a backport of Gateway_Chassis to HA_Chassis_Group patch
stack (https://review.opendev.org/c/openstack/neutron/+/947317/13)
Router with multiple Geneve and VLAN networks (plus a provider network
with gateway):
192.168.44.0/24 - VLAN type network
172.16.0.1/23 - Geneve type network
masked_internet_ip/28 - VLAN type provider network (FIP and external gateway)
ovn-nbctl show output for given router:
router 1c387a14-81d8-4277-88bb-9a8307599991 (neutron-a1970ca5-0e53-4c22-9199-6f919903335b) (aka external)
- port lrp-c4ca7855-b4ba-45c0-b963-1cdb9bc6a423
- mac: "fa:16:3e:6f:4e:b4"
- networks: ["172.16.0.1/23"]
- port lrp-7508355e-87ce-4ba4-b3e3-ebd56afeec32
- mac: "fa:16:3e:a7:78:79"
- networks: ["192.168.44.1/24"]
- port lrp-3eb71d03-923e-4434-8502-791b2df8dc0c
- mac: "fa:16:3e:19:25:a0"
- networks: ["masked_internet_ip/28", "masked_internet_ipv6/127"]
+ port lrp-c4ca7855-b4ba-45c0-b963-1cdb9bc6a423
+ mac: "fa:16:3e:6f:4e:b4"
+ networks: ["172.16.0.1/23"]
+ port lrp-7508355e-87ce-4ba4-b3e3-ebd56afeec32
+ mac: "fa:16:3e:a7:78:79"
+ networks: ["192.168.44.1/24"]
+ port lrp-3eb71d03-923e-4434-8502-791b2df8dc0c
+ mac: "fa:16:3e:19:25:a0"
+ networks: ["masked_internet_ip/28", "masked_internet_ipv6/127"]
# ovn-nbctl find Logical_Router_Port name=lrp-7508355e-87ce-4ba4-b3e3-ebd56afeec32
_uuid : ab91bd65-4830-4fcd-919a-f87aa03b06e5
enabled : []
external_ids : {"neutron:is_ext_gw"=False, "neutron:network_name"=neutron-c557a6bb-ad28-4479-972a-4e842f328d3a, "neutron:revision_number"="2", "neutron:router_name"="a1970ca5-0e53-4c22-9199-6f919903335b", "neutron:subnet_ids"="c4e30d02-b219-4c4c-85f5-7859ddc88193"}
gateway_chassis : []
ha_chassis_group : []
ipv6_prefix : []
ipv6_ra_configs : {}
mac : "fa:16:3e:a7:78:79"
name : lrp-7508355e-87ce-4ba4-b3e3-ebd56afeec32
networks : ["192.168.44.1/24"]
options : {reside-on-redirect-chassis="true"}
peer : []
status : {}
# ovn-nbctl find Logical_Router_Port name=lrp-c4ca7855-b4ba-45c0-b963-1cdb9bc6a423
_uuid : 86514488-f62a-4982-80b0-3e9e1177185a
enabled : []
external_ids : {"neutron:is_ext_gw"=False, "neutron:network_name"=neutron-e2b9e59f-43da-4e1c-b558-dc9da4c0d738, "neutron:revision_number"="2", "neutron:router_name"="a1970ca5-0e53-4c22-9199-6f919903335b", "neutron:subnet_ids"="724f792d-183f-406b-b207-02050126813f"}
gateway_chassis : []
ha_chassis_group : []
ipv6_prefix : []
ipv6_ra_configs : {}
mac : "fa:16:3e:6f:4e:b4"
name : lrp-c4ca7855-b4ba-45c0-b963-1cdb9bc6a423
networks : ["172.16.0.1/23"]
options : {}
peer : []
status : {}
# ovn-nbctl find Logical_Router_Port name=lrp-3eb71d03-923e-4434-8502-791b2df8dc0c
_uuid : 79e57ba7-b7d3-4174-a170-9341cc3210eb
enabled : []
external_ids : {"neutron:is_ext_gw"=True, "neutron:network_name"=neutron-80e70560-297d-498e-9acc-51e7822d09a8, "neutron:revision_number"="492", "neutron:router_name"="a1970ca5-0e53-4c22-9199-6f919903335b", "neutron:subnet_ids"="1674bc43-b481-4e13-ad17-c1b80144a282 616f7216-c329-419b-bf39-791b58babd74"}
gateway_chassis : []
ha_chassis_group : 9c8a379b-6a3d-4a1b-84ab-c1d6bd9096c2
ipv6_prefix : []
ipv6_ra_configs : {}
mac : "fa:16:3e:19:25:a0"
name : lrp-3eb71d03-923e-4434-8502-791b2df8dc0c
networks : ["masked_internet_ip/28", "masked_internet_ipv6/127"]
options : {reside-on-redirect-chassis="true"}
peer : []
status : {hosting-chassis=controller03}
Running ping from a VM with address 172.16.0.149 (Geneve network) to a
VM with address 192.168.44.228 gives ICMP replies in the first ,,pinging
session'':
$ ping 192.168.44.228
PING 192.168.44.228 (192.168.44.228) 56(84) bytes of data.
64 bytes from 192.168.44.228: icmp_seq=1 ttl=63 time=3.02 ms
64 bytes from 192.168.44.228: icmp_seq=2 ttl=63 time=1.40 ms
^C
--- 192.168.44.228 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 1.396/2.208/3.021/0.812 ms
And then nothing comes back in the second session:
$ ping 192.168.44.228
PING 192.168.44.228 (192.168.44.228) 56(84) bytes of data.
^C
--- 192.168.44.228 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2038ms
the ICMP reply packets are visible in tcpdump on the controller (network
gateway) - but are not reaching the VM (172.16.0.149)
After some minutes of no traffic between these hosts - the first
,,pinging session'' gets some ICMP replies, but after ctrl+c and running
it again - no replies
Removing reside-on-redirect-chassis=true from the 192.168.44.1 LRP
brings the traffic between these VMs to normal - but SNAT from
192.168.44.0/24 to the internet stops working.
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2124259
Title:
[ovn] Direct routing (non NAT) does not work between VLAN and Geneve
networks
Status in neutron:
New
Bug description:
Environment:
OpenStack Kolla-Ansible deployed 2025.1
Rocky Linux 9.6
OVN version: latest 24.03 from CentOS NFV SIG - ovn24.03-24.03.6-22.el9s.x86_64
OVS version: openvswitch3.3-3.3.4-115.el9s.x86_64
Distributed floating IP enabled
Running with a backport of Gateway_Chassis to HA_Chassis_Group patch
stack (https://review.opendev.org/c/openstack/neutron/+/947317/13)
Router with multiple Geneve and VLAN networks (plus a provider network
with gateway):
192.168.44.0/24 - VLAN type network
172.16.0.1/23 - Geneve type network
masked_internet_ip/28 - VLAN type provider network (FIP and external gateway)
ovn-nbctl show output for given router:
router 1c387a14-81d8-4277-88bb-9a8307599991 (neutron-a1970ca5-0e53-4c22-9199-6f919903335b) (aka external)
port lrp-c4ca7855-b4ba-45c0-b963-1cdb9bc6a423
mac: "fa:16:3e:6f:4e:b4"
networks: ["172.16.0.1/23"]
port lrp-7508355e-87ce-4ba4-b3e3-ebd56afeec32
mac: "fa:16:3e:a7:78:79"
networks: ["192.168.44.1/24"]
port lrp-3eb71d03-923e-4434-8502-791b2df8dc0c
mac: "fa:16:3e:19:25:a0"
networks: ["masked_internet_ip/28", "masked_internet_ipv6/127"]
# ovn-nbctl find Logical_Router_Port name=lrp-7508355e-87ce-4ba4-b3e3-ebd56afeec32
_uuid : ab91bd65-4830-4fcd-919a-f87aa03b06e5
enabled : []
external_ids : {"neutron:is_ext_gw"=False, "neutron:network_name"=neutron-c557a6bb-ad28-4479-972a-4e842f328d3a, "neutron:revision_number"="2", "neutron:router_name"="a1970ca5-0e53-4c22-9199-6f919903335b", "neutron:subnet_ids"="c4e30d02-b219-4c4c-85f5-7859ddc88193"}
gateway_chassis : []
ha_chassis_group : []
ipv6_prefix : []
ipv6_ra_configs : {}
mac : "fa:16:3e:a7:78:79"
name : lrp-7508355e-87ce-4ba4-b3e3-ebd56afeec32
networks : ["192.168.44.1/24"]
options : {reside-on-redirect-chassis="true"}
peer : []
status : {}
# ovn-nbctl find Logical_Router_Port name=lrp-c4ca7855-b4ba-45c0-b963-1cdb9bc6a423
_uuid : 86514488-f62a-4982-80b0-3e9e1177185a
enabled : []
external_ids : {"neutron:is_ext_gw"=False, "neutron:network_name"=neutron-e2b9e59f-43da-4e1c-b558-dc9da4c0d738, "neutron:revision_number"="2", "neutron:router_name"="a1970ca5-0e53-4c22-9199-6f919903335b", "neutron:subnet_ids"="724f792d-183f-406b-b207-02050126813f"}
gateway_chassis : []
ha_chassis_group : []
ipv6_prefix : []
ipv6_ra_configs : {}
mac : "fa:16:3e:6f:4e:b4"
name : lrp-c4ca7855-b4ba-45c0-b963-1cdb9bc6a423
networks : ["172.16.0.1/23"]
options : {}
peer : []
status : {}
# ovn-nbctl find Logical_Router_Port name=lrp-3eb71d03-923e-4434-8502-791b2df8dc0c
_uuid : 79e57ba7-b7d3-4174-a170-9341cc3210eb
enabled : []
external_ids : {"neutron:is_ext_gw"=True, "neutron:network_name"=neutron-80e70560-297d-498e-9acc-51e7822d09a8, "neutron:revision_number"="492", "neutron:router_name"="a1970ca5-0e53-4c22-9199-6f919903335b", "neutron:subnet_ids"="1674bc43-b481-4e13-ad17-c1b80144a282 616f7216-c329-419b-bf39-791b58babd74"}
gateway_chassis : []
ha_chassis_group : 9c8a379b-6a3d-4a1b-84ab-c1d6bd9096c2
ipv6_prefix : []
ipv6_ra_configs : {}
mac : "fa:16:3e:19:25:a0"
name : lrp-3eb71d03-923e-4434-8502-791b2df8dc0c
networks : ["masked_internet_ip/28", "masked_internet_ipv6/127"]
options : {reside-on-redirect-chassis="true"}
peer : []
status : {hosting-chassis=controller03}
Running ping from a VM with address 172.16.0.149 (Geneve network) to a
VM with address 192.168.44.228 gives ICMP replies in the first
,,pinging session'':
$ ping 192.168.44.228
PING 192.168.44.228 (192.168.44.228) 56(84) bytes of data.
64 bytes from 192.168.44.228: icmp_seq=1 ttl=63 time=3.02 ms
64 bytes from 192.168.44.228: icmp_seq=2 ttl=63 time=1.40 ms
^C
--- 192.168.44.228 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 1.396/2.208/3.021/0.812 ms
And then nothing comes back in the second session:
$ ping 192.168.44.228
PING 192.168.44.228 (192.168.44.228) 56(84) bytes of data.
^C
--- 192.168.44.228 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2038ms
the ICMP reply packets are visible in tcpdump on the controller
(network gateway) - but are not reaching the VM (172.16.0.149)
After some minutes of no traffic between these hosts - the first
,,pinging session'' gets some ICMP replies, but after ctrl+c and
running it again - no replies
Removing reside-on-redirect-chassis=true from the 192.168.44.1 LRP
brings the traffic between these VMs to normal - but SNAT from
192.168.44.0/24 to the internet stops working.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2124259/+subscriptions
