yahoo-eng-team team mailing list archive

Thread
Date

[Bug 2084536] [NEW] OVN: Connecting internal tenant VLAN networks to virtual routers

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Gabriel Samfira <2084536@xxxxxxxxxxxxxxxxxx>
Date: Tue, 15 Oct 2024 10:45:49 -0000
Reply-to: Bug 2084536 <2084536@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx

Public bug reported:

Hello folks,

As discussed on IRC, I am opening this bug to start a conversation about
the ability to use internal tenant VLAN networks with OVN. This is a
feature that exists in neutron gateway and which seems to not work with
OVN when external ports are involved.

The TL;DR version of things is as follows:

1) An internal tenant VLAN network is created
2) A subnet is added to the VLAN network
3) Create a Geneve network
4) Add a subnet to the Geneve network
5) Create a router
6) Attach an interface to the VLAN network
7) Attach an interface to the Geneve network
8) Boot an Ironic bare metal node connected to the VLAN network (we used networking-genericswitch)
9) Ping from the bare metal node to a VM in the Geneve network

This scenario is roughly described in this comment from another bug
which has the same root cause:

https://bugs.launchpad.net/neutron/+bug/1995078/comments/34

In our environment, we need to use VLAN networks for bare metal. We
would also like to be able to use internal networks, as that greatly
simplifies our setup. We don't need to create provider networks and
gateways for each tenant. We can set up a pool of VLAN ranges that they
can create networks from. Not all VLAN networks need to be external.

Currently, the issue is that internal router ports do not get bound to
any chassis when using OVN. With Neutron Gateway you get this out of the
box via the qrouter netns, where a veth gets added, but with OVN, this
does not happen.

This means that VLAN tenant networks will work if you use them on a VM,
due to the distributed port feature, but when dealing with external
(baremetal) ports, this will not work. The bare metal node will issue an
ARP request for the MAC address of the default gateway (the internal
router port) but no reply is ever sent. Exception to this rule seems to
be if the vrouter has an external port attached, and the chassis gateway
of that external port, match the same hosts and priorities of the
internal VLAN network that is also attached to the same router. But this
is not feasible, as we may have multiple internal VLAN networks
attached, rach potentially with a different Chassis added to their own
HA chassis groups.

The potential fix for this is "simple". Binding internal ports for VLAN
(and potentially flat) networks to the same HA chassis group that the
VLAN network itself belongs to, will ensure that both (baremetal)
external ports and virtual router interfaces will live on the same
chassis. Essentially, this means treating internal VLAN networks, the
same way we treat external networks. Conceptually, there is no
difference between an Ironic managed bare metal node and an upstream
gateway that needs to communicate with the external port in a virtual
router to ensure egress connectivity.

We implemented a proof-of-concept patch here:

https://review.opendev.org/c/openstack/neutron/+/931892

This allows us to use Ironic with OVN without any issues, regardless of
what networks we decide to connect to a virtual router. The alternative
to something like this is to stick with Neutron Gateway, but given the
scale of the cloud that will be created, we would rather stick with OVN
if possible.

** Affects: neutron
Importance: Undecided
Status: New

--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2084536

Title:
OVN: Connecting internal tenant VLAN networks to virtual routers

Status in neutron:
New

Bug description:
Hello folks,

As discussed on IRC, I am opening this bug to start a conversation
about the ability to use internal tenant VLAN networks with OVN. This
is a feature that exists in neutron gateway and which seems to not
work with OVN when external ports are involved.

The TL;DR version of things is as follows:

This scenario is roughly described in this comment from another bug
which has the same root cause:

https://bugs.launchpad.net/neutron/+bug/1995078/comments/34

In our environment, we need to use VLAN networks for bare metal. We
would also like to be able to use internal networks, as that greatly
simplifies our setup. We don't need to create provider networks and
gateways for each tenant. We can set up a pool of VLAN ranges that
they can create networks from. Not all VLAN networks need to be
external.

Currently, the issue is that internal router ports do not get bound to
any chassis when using OVN. With Neutron Gateway you get this out of
the box via the qrouter netns, where a veth gets added, but with OVN,
this does not happen.

This means that VLAN tenant networks will work if you use them on a
VM, due to the distributed port feature, but when dealing with
external (baremetal) ports, this will not work. The bare metal node
will issue an ARP request for the MAC address of the default gateway
(the internal router port) but no reply is ever sent. Exception to
this rule seems to be if the vrouter has an external port attached,
and the chassis gateway of that external port, match the same hosts
and priorities of the internal VLAN network that is also attached to
the same router. But this is not feasible, as we may have multiple
internal VLAN networks attached, rach potentially with a different
Chassis added to their own HA chassis groups.

The potential fix for this is "simple". Binding internal ports for
VLAN (and potentially flat) networks to the same HA chassis group that
the VLAN network itself belongs to, will ensure that both (baremetal)
external ports and virtual router interfaces will live on the same
chassis. Essentially, this means treating internal VLAN networks, the
same way we treat external networks. Conceptually, there is no
difference between an Ironic managed bare metal node and an upstream
gateway that needs to communicate with the external port in a virtual
router to ensure egress connectivity.

We implemented a proof-of-concept patch here:

https://review.opendev.org/c/openstack/neutron/+/931892

This allows us to use Ironic with OVN without any issues, regardless
of what networks we decide to connect to a virtual router. The
alternative to something like this is to stick with Neutron Gateway,
but given the scale of the cloud that will be created, we would rather
stick with OVN if possible.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2084536/+subscriptions