yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #90623
[Bug 1998609] [NEW] [RFE] - OVN Distributed routing + IPv6 support
Public bug reported:
Hi,
Currently, DVR support for IPv4 FIP addresses works perfectly with OVN.
However, I would like to request if it is possible to extend this
behavior to IPv6 addresses. When I talk about IPv6 addresses, I'm
referring to the GUA addresses that are allocated to VMs (e.g.,
considering an openstack deployment).
---------------------------------
Adding a use case context:
For DVR to be enabled properly, the provider networks must be stretched over the Underlay Network and each Compute Node would have the bridge for external traffic. In an L3 Leaf-Spine Underlay, for example, the network to be reached is for the Underlay Network be able to stretch an L2 domain(VLAN) via VXLAN as dataplane and BGP EVPN as Control Plane. In this solution, the Leaf switches would need to work as HW VTEP Gateway to initiate and terminate the VXLANs tunnels and use BGP EVPN to learn and advertise the MAC Addresses from the Compute Node’s provider network.
The common reference architecture is detailed in [picture 1] and the design for the DVR+FRR solution (IPv4/IPv6) is detailed in [picture 2].
---------------------------------
What's the problem? well, inbound traffic to a GUA address goes through the chassis where the router's external port resides.
Looking at the DVR implementation for IPv4, I see that the solution is
heavily based on the idea of NAT.
However, the OVN/OVS support the same idea of distributed routing for
both IPv4 and IPv6. After the discussion of this thread [1], now, we
know that DVR for IPv6 is supported by ovn with some special NAT rule.
To be clear, we need to insert a NAT rule for the GUA addresses that are
allocated to VMs (ovn/ovs understands IPv6 GUA as a FIP). Even though it
is a global address, the ovn-controller running on the chassis needs
this rule to start responding GARPs (IPv4 - FIP) and neighbor
advertisements (for IPv6).
Any ideas on how best to integrate this into neutron? a NAT rule for the
VM logical_port with external ip equal to the internal ip (IPv6 GUA<->
IPv6 GUA).
In this case, we need to add a rule like this:
# ovn-nbctl lr-nat-add <logical_router> dnat_and_snat <VM GUA> <VM GUA>
<VM logical port> <SOME CHOSEN RANDOM MAC>
The EXTERNAL_IP and LOGICAL_IP used in the NAT rule are the same (ie VM
GUA). If we add an entry like this, ovn should add logical flows to
respond to IPv6 NS requests for the VM GUA.
For example:
The VM has the IPv6 GUA => 2001:db8:1234::140
Some steps to make it work with OVN.
1 - List logical port of the VM
#ovn-sbctl list port_binding
_uuid : 31b752fe-2c1a-4fc1-9ddb-62e375faddc0
chassis : b68e7803-4117-475d-89af-7f9302027c6c
datapath : 46a6e556-8dbd-4e48-85f3-d8c7d28ae984
encap : []
external_ids : {"neutron:cidrs"="192.168.0.120/24 2001:db8:1234::140/64", "neutron:device_id"="3d2dba15-7b00-4e88-b0dc-29f009699fc2", "neutron:device_owner"="compute:nova", "neutron:network_name"=neutron-a01af33c-18f2-4706-b20a-807219967649, "neutron:port_fip"="200.201.0.226", "neutron:port_name"="", "neutron:project_id"=d11daecfe9d847ddb7d9ce2932c2fe26, "neutron:revision_number"="6", "neutron:security_group_ids"="cf2e7d53-0db7-4873-82ab-cf67eceda937"}
gateway_chassis : []
ha_chassis_group : []
logical_port : "ae528d1a-d092-4a8e-8a9a-83b849cac0a2"
mac : ["fa:16:3e:c4:15:40 192.168.0.120 2001:db8:1234::140"]
nat_addresses : []
options : {mcast_flood_reports="true", requested-chassis=compute2}
parent_port : []
requested_chassis : b68e7803-4117-475d-89af-7f9302027c6c
tag : []
tunnel_key : 4
type : ""
up : true
virtual_parent : []
logical_port : "ae528d1a-d092-4a8e-8a9a-83b849cac0a2"
2 - Discovery the logical router id
#ovn-nbctl list logical_router
_uuid : 84afed40-9bb7-4579-ad9c-2392f5398635
copp : []
enabled : true
external_ids : {"neutron:availability_zone_hints"="", "neutron:gw_port_id"="bcf42b8c-5ca5-44d1-8065-6f36892a1473", "neutron:revision_number"="4", "neutron:router_name"=router1}
load_balancer : []
load_balancer_group : []
name : neutron-a65dc3c7-295d-4602-aa3e-ae0271ae80e9
nat : [1ce157ff-8eff-4472-b9de-07c7869dd427, 5e41244f-a976-4310-bba7-ccef995efff1, c0ea96d4-8dc6-4b0a-9bee-9d1484500362, eea84cbd-3106-4637-abfd-ce1b592d0f7d, f01d7117-f2be-489d-a374-652aa510d629]
options : {always_learn_from_arp_request="false", dynamic_neigh_routers="true"}
policies : []
ports : [2c0e31ff-e4a0-4638-81de-bfc786051ea4, 3e87914b-f7f4-4a11-8a1a-c3ef92e956a7, 4dc352cb-7fcc-42ca-a99a-9bdc73c54c6d]
static_routes : [7159024a-9c36-43c8-b62a-4f56efe66fb3, 950a419a-683e-4ea6-bd5a-be16d61a39ac]
3 - Insert the new NAT rule for GUA, like this:
ovn-nbctl lr-nat-add 84afed40-9bb7-4579-ad9c-2392f5398635 dnat_and_snat 2001:db8:1234::140 2001:db8:1234::140 ae528d1a-d092-4a8e-8a9a-83b849cac0a2 fa:16:3e:c4:aa:bb
In this example we use a random MAC fa:16:3e:c4:aa:bb
After that, we can see that the neighbor solicitation/neighbor advertisement and ICMPv6 packets pass through the correct chassis interface directly (without forwarding via geneve to the other compute - where the external router port for the provider network is installed)
Thanks,
Roberto
[picture 1]- https://drive.google.com/file/d/1oaGmKbFGHqMwBxKVxsT4I-7rFt2pXSJW/view?usp=sharing
[picture 2]- https://drive.google.com/file/d/1E-MRe9WJubPz5ZP836bNPFPRMuPCt4_s/view?usp=sharing
[1]-https://mail.openvswitch.org/pipermail/ovs-
discuss/2022-December/052126.html
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1998609
Title:
[RFE] - OVN Distributed routing + IPv6 support
Status in neutron:
New
Bug description:
Hi,
Currently, DVR support for IPv4 FIP addresses works perfectly with
OVN. However, I would like to request if it is possible to extend this
behavior to IPv6 addresses. When I talk about IPv6 addresses, I'm
referring to the GUA addresses that are allocated to VMs (e.g.,
considering an openstack deployment).
---------------------------------
Adding a use case context:
For DVR to be enabled properly, the provider networks must be stretched over the Underlay Network and each Compute Node would have the bridge for external traffic. In an L3 Leaf-Spine Underlay, for example, the network to be reached is for the Underlay Network be able to stretch an L2 domain(VLAN) via VXLAN as dataplane and BGP EVPN as Control Plane. In this solution, the Leaf switches would need to work as HW VTEP Gateway to initiate and terminate the VXLANs tunnels and use BGP EVPN to learn and advertise the MAC Addresses from the Compute Node’s provider network.
The common reference architecture is detailed in [picture 1] and the design for the DVR+FRR solution (IPv4/IPv6) is detailed in [picture 2].
---------------------------------
What's the problem? well, inbound traffic to a GUA address goes through the chassis where the router's external port resides.
Looking at the DVR implementation for IPv4, I see that the solution is
heavily based on the idea of NAT.
However, the OVN/OVS support the same idea of distributed routing for
both IPv4 and IPv6. After the discussion of this thread [1], now, we
know that DVR for IPv6 is supported by ovn with some special NAT rule.
To be clear, we need to insert a NAT rule for the GUA addresses that
are allocated to VMs (ovn/ovs understands IPv6 GUA as a FIP). Even
though it is a global address, the ovn-controller running on the
chassis needs this rule to start responding GARPs (IPv4 - FIP) and
neighbor advertisements (for IPv6).
Any ideas on how best to integrate this into neutron? a NAT rule for
the VM logical_port with external ip equal to the internal ip (IPv6
GUA<-> IPv6 GUA).
In this case, we need to add a rule like this:
# ovn-nbctl lr-nat-add <logical_router> dnat_and_snat <VM GUA> <VM
GUA> <VM logical port> <SOME CHOSEN RANDOM MAC>
The EXTERNAL_IP and LOGICAL_IP used in the NAT rule are the same (ie
VM GUA). If we add an entry like this, ovn should add logical flows to
respond to IPv6 NS requests for the VM GUA.
For example:
The VM has the IPv6 GUA => 2001:db8:1234::140
Some steps to make it work with OVN.
1 - List logical port of the VM
#ovn-sbctl list port_binding
_uuid : 31b752fe-2c1a-4fc1-9ddb-62e375faddc0
chassis : b68e7803-4117-475d-89af-7f9302027c6c
datapath : 46a6e556-8dbd-4e48-85f3-d8c7d28ae984
encap : []
external_ids : {"neutron:cidrs"="192.168.0.120/24 2001:db8:1234::140/64", "neutron:device_id"="3d2dba15-7b00-4e88-b0dc-29f009699fc2", "neutron:device_owner"="compute:nova", "neutron:network_name"=neutron-a01af33c-18f2-4706-b20a-807219967649, "neutron:port_fip"="200.201.0.226", "neutron:port_name"="", "neutron:project_id"=d11daecfe9d847ddb7d9ce2932c2fe26, "neutron:revision_number"="6", "neutron:security_group_ids"="cf2e7d53-0db7-4873-82ab-cf67eceda937"}
gateway_chassis : []
ha_chassis_group : []
logical_port : "ae528d1a-d092-4a8e-8a9a-83b849cac0a2"
mac : ["fa:16:3e:c4:15:40 192.168.0.120 2001:db8:1234::140"]
nat_addresses : []
options : {mcast_flood_reports="true", requested-chassis=compute2}
parent_port : []
requested_chassis : b68e7803-4117-475d-89af-7f9302027c6c
tag : []
tunnel_key : 4
type : ""
up : true
virtual_parent : []
logical_port : "ae528d1a-d092-4a8e-8a9a-83b849cac0a2"
2 - Discovery the logical router id
#ovn-nbctl list logical_router
_uuid : 84afed40-9bb7-4579-ad9c-2392f5398635
copp : []
enabled : true
external_ids : {"neutron:availability_zone_hints"="", "neutron:gw_port_id"="bcf42b8c-5ca5-44d1-8065-6f36892a1473", "neutron:revision_number"="4", "neutron:router_name"=router1}
load_balancer : []
load_balancer_group : []
name : neutron-a65dc3c7-295d-4602-aa3e-ae0271ae80e9
nat : [1ce157ff-8eff-4472-b9de-07c7869dd427, 5e41244f-a976-4310-bba7-ccef995efff1, c0ea96d4-8dc6-4b0a-9bee-9d1484500362, eea84cbd-3106-4637-abfd-ce1b592d0f7d, f01d7117-f2be-489d-a374-652aa510d629]
options : {always_learn_from_arp_request="false", dynamic_neigh_routers="true"}
policies : []
ports : [2c0e31ff-e4a0-4638-81de-bfc786051ea4, 3e87914b-f7f4-4a11-8a1a-c3ef92e956a7, 4dc352cb-7fcc-42ca-a99a-9bdc73c54c6d]
static_routes : [7159024a-9c36-43c8-b62a-4f56efe66fb3, 950a419a-683e-4ea6-bd5a-be16d61a39ac]
3 - Insert the new NAT rule for GUA, like this:
ovn-nbctl lr-nat-add 84afed40-9bb7-4579-ad9c-2392f5398635 dnat_and_snat 2001:db8:1234::140 2001:db8:1234::140 ae528d1a-d092-4a8e-8a9a-83b849cac0a2 fa:16:3e:c4:aa:bb
In this example we use a random MAC fa:16:3e:c4:aa:bb
After that, we can see that the neighbor solicitation/neighbor advertisement and ICMPv6 packets pass through the correct chassis interface directly (without forwarding via geneve to the other compute - where the external router port for the provider network is installed)
Thanks,
Roberto
[picture 1]- https://drive.google.com/file/d/1oaGmKbFGHqMwBxKVxsT4I-7rFt2pXSJW/view?usp=sharing
[picture 2]- https://drive.google.com/file/d/1E-MRe9WJubPz5ZP836bNPFPRMuPCt4_s/view?usp=sharing
[1]-https://mail.openvswitch.org/pipermail/ovs-
discuss/2022-December/052126.html
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1998609/+subscriptions
