← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #96134

[Bug 2116249] [NEW] OVN network metadata is broken if allowed-address is configured using the same IP

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

Bug was originally reported by Mathew Flusche at
https://issues.redhat.com/browse/OSPRH-17820 for Red Hat Openstack but I
confirmed it happens also on the upstream OpenStack with master branch.

Original description:

Setting the subnet's dhcp/metadata IP as an allowed_address for a port
causes a metadata outage.

To Reproduce Steps to reproduce the behavior:
{{openstack network create net1
openstack subnet create --subnet-range 10.1.1.0/24 --network net1 subnet1
openstack router create router1
openstack router add subnet router1 subnet1
openstack router set --external-gateway external2 router1
openstack server create --image rhel8 --key-name default --network net1 --flavor rhel vm3
openstack floating ip create external2
openstack server add floating ip vm3 192.168.2.176

    Verify metadata works fine.

$ ssh cloud-user@192.168.2.176
[cloud-user@vm3 ~]$ curl 169.254.169.254
1.0
2007-01-19
2007-03-01
2007-08-29
2007-10-10
2007-12-15
2008-02-01
2008-09-01
2009-04-04

In OVN, verify metadata port is correctly set as type: localport

openstack port list --device-owner network:distributed --network net1
----------------------------------------------------------------------------------------------------------------------------------------
ID 	Name 	MAC Address 	Fixed IP Addresses 	Status

----------------------------------------------------------------------------------------------------------------------------------------
7103f027-0736-49c2-804d-50ecbc8329ed 	  	fa:16:3e:32:0a:81 	ip_address='10.1.1.2', subnet_id='d2f5d2dc-2814-4848-b3e2-bbfadf375046' 	DOWN

----------------------------------------------------------------------------------------------------------------------------------------

ovn-sbctl list port_binding 7103f027-0736-49c2-804d-50ecbc8329ed |grep ^type
type : localport

All good, delete the test instance (so metadata proxy is removed for the
net):

openstack server delete vm3

    Create test port and allowed-address that will break ovn metadata.

openstack port create --network net1 testport
openstack port set --allowed-address ip-address=10.1.1.2 testport

    Create VM instance and verify metadata is broken.

openstack server create --image rhel8 --key-name default --network net1 --flavor rhel vm3
openstack server add floating ip vm3 192.168.2.176

[cloud-user@vm3 ~]$ curl -v 169.254.169.254

    Rebuilt URL to: 169.254.169.254/
    Trying 169.254.169.254...
    TCP_NODELAY set
    connect to 169.254.169.254 port 80 failed: No route to host
    Failed to connect to 169.254.169.254 port 80: No route to host
    Closing connection 0
    curl: (7) Failed to connect to 169.254.169.254 port 80: No route to host

    Verify in OVN the metadata port is now type: virtual and has a
virtual parent pointing the test port.

ovn-sbctl list port_binding 7103f027-0736-49c2-804d-50ecbc8329ed |egrep '^type|^options'
options :
{requested-chassis="", virtual-ip="10.1.1.2", virtual-parents="429f0583-0eae-459b-b4a2-004d26c88485"}

type : virtual

In the metadata agent debug logs we see the following indicating the
misconfigured metadata port:

Jun 27 13:59:45 compute18-node2 ovn_metadata_agent[2307]: 2025-06-27 13:59:45.833 2473 INFO neutron.agent.ovn.metadata.agent [-] Port da7b948e-865a-4a12-805b-23684f
8fa208 in datapath 30eba16f-5e67-44d5-9560-7f49a60d9d85 bound to our chassis
Jun 27 13:59:45 compute18-node2 ovn_metadata_agent[2307]: 2025-06-27 13:59:45.834 2473 DEBUG neutron.agent.ovn.metadata.agent [-] There is no metadata port for netw
ork 30eba16f-5e67-44d5-9560-7f49a60d9d85 or it has no MAC address configured, tearing the namespace down if needed _get_provision_params /usr/lib/python3.9/site-pac
kages/neutron/agent/ovn/metadata/agent.py:682

    To resolve the issue, remove the allowed-address and trigger the
port reconfig by removing/adding dhcp to the network.

openstack port set --no-allowed-address testport
openstack subnet set --no-dhcp subnet1
openstack subnet set --dhcp subnet1

    Port type is fixed and curl works from VM:

ovn-sbctl list port_binding 7103f027-0736-49c2-804d-50ecbc8329ed |egrep '^type|^options'
options :
{requested-chassis=""}

type : localport

[cloud-user@vm3 ~]$ curl 169.254.169.254
1.0
2007-01-19
2007-03-01
2007-08-29
2007-10-10
2007-12-15
2008-02-01
2008-09-01
2009-04-04
}}

Expected behavior
Network metadata should have priority and the allowed-address config should fail.

Bug impact
Metadata outage for impacted network

Additional context
This issue was found on an OSP 16.2 environment but reproduces on RHOSO 18 also.

** Affects: neutron
     Importance: Medium
     Assignee: Slawek Kaplonski (slaweq)
         Status: Confirmed


** Tags: ovn

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2116249

Title:
  OVN network metadata is broken if allowed-address is configured using
  the same IP

Status in neutron:
  Confirmed

Bug description:
  Bug was originally reported by Mathew Flusche at
  https://issues.redhat.com/browse/OSPRH-17820 for Red Hat Openstack but
  I confirmed it happens also on the upstream OpenStack with master
  branch.

  Original description:

  Setting the subnet's dhcp/metadata IP as an allowed_address for a port
  causes a metadata outage.

  To Reproduce Steps to reproduce the behavior:
  {{openstack network create net1
  openstack subnet create --subnet-range 10.1.1.0/24 --network net1 subnet1
  openstack router create router1
  openstack router add subnet router1 subnet1
  openstack router set --external-gateway external2 router1
  openstack server create --image rhel8 --key-name default --network net1 --flavor rhel vm3
  openstack floating ip create external2
  openstack server add floating ip vm3 192.168.2.176

      Verify metadata works fine.

  $ ssh cloud-user@192.168.2.176
  [cloud-user@vm3 ~]$ curl 169.254.169.254
  1.0
  2007-01-19
  2007-03-01
  2007-08-29
  2007-10-10
  2007-12-15
  2008-02-01
  2008-09-01
  2009-04-04

  In OVN, verify metadata port is correctly set as type: localport

  openstack port list --device-owner network:distributed --network net1
  ----------------------------------------------------------------------------------------------------------------------------------------
  ID 	Name 	MAC Address 	Fixed IP Addresses 	Status

  ----------------------------------------------------------------------------------------------------------------------------------------
  7103f027-0736-49c2-804d-50ecbc8329ed 	  	fa:16:3e:32:0a:81 	ip_address='10.1.1.2', subnet_id='d2f5d2dc-2814-4848-b3e2-bbfadf375046' 	DOWN

  ----------------------------------------------------------------------------------------------------------------------------------------

  ovn-sbctl list port_binding 7103f027-0736-49c2-804d-50ecbc8329ed |grep ^type
  type : localport

  All good, delete the test instance (so metadata proxy is removed for
  the net):

  openstack server delete vm3

      Create test port and allowed-address that will break ovn metadata.

  openstack port create --network net1 testport
  openstack port set --allowed-address ip-address=10.1.1.2 testport

      Create VM instance and verify metadata is broken.

  openstack server create --image rhel8 --key-name default --network net1 --flavor rhel vm3
  openstack server add floating ip vm3 192.168.2.176

  [cloud-user@vm3 ~]$ curl -v 169.254.169.254

      Rebuilt URL to: 169.254.169.254/
      Trying 169.254.169.254...
      TCP_NODELAY set
      connect to 169.254.169.254 port 80 failed: No route to host
      Failed to connect to 169.254.169.254 port 80: No route to host
      Closing connection 0
      curl: (7) Failed to connect to 169.254.169.254 port 80: No route to host

      Verify in OVN the metadata port is now type: virtual and has a
  virtual parent pointing the test port.

  ovn-sbctl list port_binding 7103f027-0736-49c2-804d-50ecbc8329ed |egrep '^type|^options'
  options :
  {requested-chassis="", virtual-ip="10.1.1.2", virtual-parents="429f0583-0eae-459b-b4a2-004d26c88485"}

  type : virtual

  In the metadata agent debug logs we see the following indicating the
  misconfigured metadata port:

  Jun 27 13:59:45 compute18-node2 ovn_metadata_agent[2307]: 2025-06-27 13:59:45.833 2473 INFO neutron.agent.ovn.metadata.agent [-] Port da7b948e-865a-4a12-805b-23684f
  8fa208 in datapath 30eba16f-5e67-44d5-9560-7f49a60d9d85 bound to our chassis
  Jun 27 13:59:45 compute18-node2 ovn_metadata_agent[2307]: 2025-06-27 13:59:45.834 2473 DEBUG neutron.agent.ovn.metadata.agent [-] There is no metadata port for netw
  ork 30eba16f-5e67-44d5-9560-7f49a60d9d85 or it has no MAC address configured, tearing the namespace down if needed _get_provision_params /usr/lib/python3.9/site-pac
  kages/neutron/agent/ovn/metadata/agent.py:682

      To resolve the issue, remove the allowed-address and trigger the
  port reconfig by removing/adding dhcp to the network.

  openstack port set --no-allowed-address testport
  openstack subnet set --no-dhcp subnet1
  openstack subnet set --dhcp subnet1

      Port type is fixed and curl works from VM:

  ovn-sbctl list port_binding 7103f027-0736-49c2-804d-50ecbc8329ed |egrep '^type|^options'
  options :
  {requested-chassis=""}

  type : localport

  [cloud-user@vm3 ~]$ curl 169.254.169.254
  1.0
  2007-01-19
  2007-03-01
  2007-08-29
  2007-10-10
  2007-12-15
  2008-02-01
  2008-09-01
  2009-04-04
  }}

  Expected behavior
  Network metadata should have priority and the allowed-address config should fail.

  Bug impact
  Metadata outage for impacted network

  Additional context
  This issue was found on an OSP 16.2 environment but reproduces on RHOSO 18 also.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2116249/+subscriptions
  Thread Previous • Date Previous • • Thread Next