yahoo-eng-team team mailing list archive

[Carl Baldwin <carl.baldwin@xxxxxx>]

Public bug reported:

This subject came up on IRC here [1]. It relates to the blueprint about
pluggable external network connections and so I jumped in.

There are two reasons that using multiple external networks to allow multiple floating ip subnets [2] is not optimal.
- Extra L2 infrastructure needed.
- A neutron router cannot have a gateway connection to more than one external network. So, floating IPs wouldn't be able to float as freely as we'd like them to.

I cracked open devstack and started playing with it. I tried this first
just to add a second subnet full of floating IPs.

neutron subnet-create ext-net 10.224.24.0/24 --disable-dhcp

In devstack, I needed to add a "gateway router". I did this by adding an
IP to the br-ex interface. In public cloud, we'd need to configure the
upstream router as a gateway on the second subnet. This shouldn't be
difficult. We'd have to run this by the networking team to be sure.

sudo ip addr add 10.224.24.1/24 dev br-ex

At this point, I was able to get a router to host floating IPs on both
subnets! Pretty cool! I was very surprised it worked so easily.

There is one bug which this bug report addresses! Traffic between
floating IPs on the second subnet went up to the router and then back
down. The upstream router sent ICMP redirect packets periodically back
to the Neutron router sourcing the traffic. These did the router no good
because what it really needed to know was that the IP was on link but
the upstream router couldn't tell it that.  Some upstream routers may
not be configured to send redirects or route back through the port of
origin.

The answer to this is to add an on-link route for each subnet on the
external network to each router's gateway interface. This will require
an L3 agent change but should not be very difficult.

[1] http://eavesdrop.openstack.org/irclogs/%23openstack-neutron/%23openstack-neutron.2014-04-08.log starting at 2014-04-08T23:23:51 (near the bottom)
[2] http://docs.openstack.org/admin-guide-cloud/content/adv_cfg_l3_agent_multi_extnet.html

** Affects: neutron
     Importance: Undecided
         Status: New

** Summary changed:

- On an external network with multiple subnets, routers need online routes for all subnets
+ On external networks with multiple subnets, routers need onlink routes for all subnets

** Description changed:

- This subject came up on IRC here http://eavesdrop.openstack.org/irclogs
- /%23openstack-neutron/%23openstack-neutron.2014-04-08.log starting at
- 2014-04-08T23:23:51 (near the bottom). It relates to the blueprint about
+ This subject came up on IRC here [1]. It relates to the blueprint about
  pluggable external network connections and so I jumped in.
  
- There are two reasons that using multiple external networks to allow multiple floating ip subnets is not optimal
+ There are two reasons that using multiple external networks to allow multiple floating ip subnets [2] is not optimal.
  - Extra L2 infrastructure needed.
  - A neutron router cannot have a gateway connection to more than one external network. So, floating IPs wouldn't be able to float as freely as we'd like them to.
  
- Since I needed to understand the current state of things for my
- blueprint, I cracked open devstack and started playing with it. I tried
- this first just to add a second subnet full of floating IPs.
+ I cracked open devstack and started playing with it. I tried this first
+ just to add a second subnet full of floating IPs.
  
  neutron subnet-create ext-net 10.224.24.0/24 --disable-dhcp
  
  In devstack, I needed to add a "gateway router". I did this by adding an
  IP to the br-ex interface. In public cloud, we'd need to configure the
  upstream router as a gateway on the second subnet. This shouldn't be
  difficult. We'd have to run this by the networking team to be sure.
  
  sudo ip addr add 10.224.24.1/24 dev br-ex
  
  At this point, I was able to get a router to host floating IPs on both
  subnets! Pretty cool! I was very surprised it worked so easily.
  
- There is one caveat! Traffic between floating IPs on the second subnet
- went up to the router and then back down. The upstream router sent ICMP
- redirect packets periodically back to the Neutron router sourcing the
- traffic. These did the router no good because what it really needed to
- know was that the IP was on link but the upstream router couldn't tell
- it that.  Some upstream routers may not be configured to send redirects
- or route back through the port of origin.
+ There is one bug which this bug report addresses! Traffic between
+ floating IPs on the second subnet went up to the router and then back
+ down. The upstream router sent ICMP redirect packets periodically back
+ to the Neutron router sourcing the traffic. These did the router no good
+ because what it really needed to know was that the IP was on link but
+ the upstream router couldn't tell it that.  Some upstream routers may
+ not be configured to send redirects or route back through the port of
+ origin.
  
  The answer to this is to add an on-link route for each subnet on the
  external network to each router's gateway interface. This will require
  an L3 agent change but should not be very difficult.
+ 
+ [1] http://eavesdrop.openstack.org/irclogs/%23openstack-neutron/%23openstack-neutron.2014-04-08.log starting at 2014-04-08T23:23:51 (near the bottom)
+ [2] http://docs.openstack.org/admin-guide-cloud/content/adv_cfg_l3_agent_multi_extnet.html

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1312467

Title:
  On external networks with multiple subnets, routers need onlink routes
  for all subnets

Status in OpenStack Neutron (virtual network service):
  New

Bug description:
  This subject came up on IRC here [1]. It relates to the blueprint
  about pluggable external network connections and so I jumped in.

  There are two reasons that using multiple external networks to allow multiple floating ip subnets [2] is not optimal.
  - Extra L2 infrastructure needed.
  - A neutron router cannot have a gateway connection to more than one external network. So, floating IPs wouldn't be able to float as freely as we'd like them to.

  I cracked open devstack and started playing with it. I tried this
  first just to add a second subnet full of floating IPs.

  neutron subnet-create ext-net 10.224.24.0/24 --disable-dhcp

  In devstack, I needed to add a "gateway router". I did this by adding
  an IP to the br-ex interface. In public cloud, we'd need to configure
  the upstream router as a gateway on the second subnet. This shouldn't
  be difficult. We'd have to run this by the networking team to be sure.

  sudo ip addr add 10.224.24.1/24 dev br-ex

  At this point, I was able to get a router to host floating IPs on both
  subnets! Pretty cool! I was very surprised it worked so easily.

  There is one bug which this bug report addresses! Traffic between
  floating IPs on the second subnet went up to the router and then back
  down. The upstream router sent ICMP redirect packets periodically back
  to the Neutron router sourcing the traffic. These did the router no
  good because what it really needed to know was that the IP was on link
  but the upstream router couldn't tell it that.  Some upstream routers
  may not be configured to send redirects or route back through the port
  of origin.

  The answer to this is to add an on-link route for each subnet on the
  external network to each router's gateway interface. This will require
  an L3 agent change but should not be very difficult.

  [1] http://eavesdrop.openstack.org/irclogs/%23openstack-neutron/%23openstack-neutron.2014-04-08.log starting at 2014-04-08T23:23:51 (near the bottom)
  [2] http://docs.openstack.org/admin-guide-cloud/content/adv_cfg_l3_agent_multi_extnet.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1312467/+subscriptions