← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #91388

[Bug 2008270] [NEW] Neutron allows you to delete router_ha_interface ports, which can lead to issues

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

We ran into a problem with a customer when some external integration
tries to remove all ports using the neutron API, including router prots.

It seems only the router ports with the router_ha_interface  device
owner are allowed to delete, all other router ports cannot be deleted
directly through the API.

Here is a simple example that demonstrates the doubling of ARP responses
if such a port is deleted:

[root@dev0 ~]# openstack router create r1 --ha --external-gateway public -c id
+-------+--------------------------------------+
| Field | Value                                |
+-------+--------------------------------------+
| id    | 5d9d6fee-6652-4843-9f7c-54c11899d721 |
+-------+--------------------------------------+
[root@dev0 ~]# neutron l3-agent-list-hosting-router r1
neutron CLI is deprecated and will be removed in the Z cycle. Use openstack CLI instead.
+--------------------------------------+------+----------------+-------+----------+
| id                                   | host | admin_state_up | alive | ha_state |
+--------------------------------------+------+----------------+-------+----------+
| 9dd0920a-cb0c-47f1-a976-3e208e3e2e6c | dev0 | True           | :-)   | active   |
| 6fa92056-ca25-42e0-aee4-c4e744008239 | dev2 | True           | :-)   | standby  |
| 8fbda128-dc9c-4b3b-be1b-bb3f11ad1447 | dev1 | True           | :-)   | standby  |
+--------------------------------------+------+----------------+-------+----------+
[root@dev0 ~]# openstack port list --device-id 5d9d6fee-6652-4843-9f7c-54c11899d721 -c id -c device_owner -c fixed_ips --long
+--------------------------------------+-----------------------------+--------------------------------------------------------------------------------+
| ID                                   | Device Owner                | Fixed IP Addresses                                                             |
+--------------------------------------+-----------------------------+--------------------------------------------------------------------------------+
| 555a9272-c9df-4a05-9f08-752c91c5a4c9 | network:router_ha_interface | ip_address='169.254.192.147', subnet_id='20c159f7-13f8-4093-9a4a-8380bdcfea60' |
| 6a196ff7-f3d4-4bee-aed0-b5d7ba727741 | network:router_ha_interface | ip_address='169.254.193.243', subnet_id='20c159f7-13f8-4093-9a4a-8380bdcfea60' |
| 7a849dcc-eac4-4d5b-a547-7ce3986ffb95 | network:router_ha_interface | ip_address='169.254.192.155', subnet_id='20c159f7-13f8-4093-9a4a-8380bdcfea60' |
| d77e624d-87a2-4135-9118-3d8e78539cee | network:router_gateway      | ip_address='10.136.17.172', subnet_id='ee15c548-e497-449e-b46d-50e9ccc0f70c'   |
+--------------------------------------+-----------------------------+--------------------------------------------------------------------------------+
[root@dev0 ~]#

[root@dev0 ~]# ip netns exec snat-5d9d6fee-6652-4843-9f7c-54c11899d721 ip a
...
25: ha-555a9272-c9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:7d:cf:a0 brd ff:ff:ff:ff:ff:ff
    inet 169.254.192.147/18 brd 169.254.255.255 scope global ha-555a9272-c9
       valid_lft forever preferred_lft forever
    inet 169.254.0.189/24 scope global ha-555a9272-c9
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe7d:cfa0/64 scope link
       valid_lft forever preferred_lft forever
28: qg-d77e624d-87: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:a8:54:29 brd ff:ff:ff:ff:ff:ff
    inet 10.136.17.172/20 scope global qg-d77e624d-87
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fea8:5429/64 scope link nodad
       valid_lft forever preferred_lft forever
[root@dev0 ~]#

[root@dev0 ~]# openstack port delete 555a9272-c9df-4a05-9f08-752c91c5a4c9
[root@dev0 ~]# neutron l3-agent-list-hosting-router r1
neutron CLI is deprecated and will be removed in the Z cycle. Use openstack CLI instead.
+--------------------------------------+------+----------------+-------+----------+
| id                                   | host | admin_state_up | alive | ha_state |
+--------------------------------------+------+----------------+-------+----------+
| 6fa92056-ca25-42e0-aee4-c4e744008239 | dev2 | True           | :-)   | active   |
| 8fbda128-dc9c-4b3b-be1b-bb3f11ad1447 | dev1 | True           | :-)   | standby  |
+--------------------------------------+------+----------------+-------+----------+
[root@dev0 ~]#

[root@dev0 ~]# ip netns exec snat-5d9d6fee-6652-4843-9f7c-54c11899d721 ip a s qg-d77e624d-87
28: qg-d77e624d-87: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:a8:54:29 brd ff:ff:ff:ff:ff:ff
    inet 10.136.17.172/20 scope global qg-d77e624d-87
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fea8:5429/64 scope link nodad
       valid_lft forever preferred_lft forever
[root@dev0 ~]# ssh dev2 ip netns exec snat-5d9d6fee-6652-4843-9f7c-54c11899d721 ip a s qg-d77e624d-87
28: qg-d77e624d-87: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:a8:54:29 brd ff:ff:ff:ff:ff:ff
    inet 10.136.17.172/20 scope global qg-d77e624d-87
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fea8:5429/64 scope link nodad
       valid_lft forever preferred_lft forever
[root@dev0 ~]#

[root@dev0 ~]# arping -c 1 -I eth0 10.136.17.172
ARPING 10.136.17.172 from 10.136.20.188 eth0
Unicast reply from 10.136.17.172 [FA:16:3E:A8:54:29]  1.537ms
Unicast reply from 10.136.17.172 [FA:16:3E:A8:54:29]  2.383ms
Sent 1 probes (1 broadcast(s))
Received 2 response(s)
[root@dev0 ~]#


As you can see, after deleting the HA port, we got a doubling of the ARP responses, which can lead to further problems in the roiting.

** Affects: neutron
     Importance: Undecided
         Status: In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2008270

Title:
  Neutron allows you to delete router_ha_interface ports, which can lead
  to issues

Status in neutron:
  In Progress

Bug description:
  We ran into a problem with a customer when some external integration
  tries to remove all ports using the neutron API, including router
  prots.

  It seems only the router ports with the router_ha_interface  device
  owner are allowed to delete, all other router ports cannot be deleted
  directly through the API.

  Here is a simple example that demonstrates the doubling of ARP
  responses if such a port is deleted:

  [root@dev0 ~]# openstack router create r1 --ha --external-gateway public -c id
  +-------+--------------------------------------+
  | Field | Value                                |
  +-------+--------------------------------------+
  | id    | 5d9d6fee-6652-4843-9f7c-54c11899d721 |
  +-------+--------------------------------------+
  [root@dev0 ~]# neutron l3-agent-list-hosting-router r1
  neutron CLI is deprecated and will be removed in the Z cycle. Use openstack CLI instead.
  +--------------------------------------+------+----------------+-------+----------+
  | id                                   | host | admin_state_up | alive | ha_state |
  +--------------------------------------+------+----------------+-------+----------+
  | 9dd0920a-cb0c-47f1-a976-3e208e3e2e6c | dev0 | True           | :-)   | active   |
  | 6fa92056-ca25-42e0-aee4-c4e744008239 | dev2 | True           | :-)   | standby  |
  | 8fbda128-dc9c-4b3b-be1b-bb3f11ad1447 | dev1 | True           | :-)   | standby  |
  +--------------------------------------+------+----------------+-------+----------+
  [root@dev0 ~]# openstack port list --device-id 5d9d6fee-6652-4843-9f7c-54c11899d721 -c id -c device_owner -c fixed_ips --long
  +--------------------------------------+-----------------------------+--------------------------------------------------------------------------------+
  | ID                                   | Device Owner                | Fixed IP Addresses                                                             |
  +--------------------------------------+-----------------------------+--------------------------------------------------------------------------------+
  | 555a9272-c9df-4a05-9f08-752c91c5a4c9 | network:router_ha_interface | ip_address='169.254.192.147', subnet_id='20c159f7-13f8-4093-9a4a-8380bdcfea60' |
  | 6a196ff7-f3d4-4bee-aed0-b5d7ba727741 | network:router_ha_interface | ip_address='169.254.193.243', subnet_id='20c159f7-13f8-4093-9a4a-8380bdcfea60' |
  | 7a849dcc-eac4-4d5b-a547-7ce3986ffb95 | network:router_ha_interface | ip_address='169.254.192.155', subnet_id='20c159f7-13f8-4093-9a4a-8380bdcfea60' |
  | d77e624d-87a2-4135-9118-3d8e78539cee | network:router_gateway      | ip_address='10.136.17.172', subnet_id='ee15c548-e497-449e-b46d-50e9ccc0f70c'   |
  +--------------------------------------+-----------------------------+--------------------------------------------------------------------------------+
  [root@dev0 ~]#

  [root@dev0 ~]# ip netns exec snat-5d9d6fee-6652-4843-9f7c-54c11899d721 ip a
  ...
  25: ha-555a9272-c9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000
      link/ether fa:16:3e:7d:cf:a0 brd ff:ff:ff:ff:ff:ff
      inet 169.254.192.147/18 brd 169.254.255.255 scope global ha-555a9272-c9
         valid_lft forever preferred_lft forever
      inet 169.254.0.189/24 scope global ha-555a9272-c9
         valid_lft forever preferred_lft forever
      inet6 fe80::f816:3eff:fe7d:cfa0/64 scope link
         valid_lft forever preferred_lft forever
  28: qg-d77e624d-87: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
      link/ether fa:16:3e:a8:54:29 brd ff:ff:ff:ff:ff:ff
      inet 10.136.17.172/20 scope global qg-d77e624d-87
         valid_lft forever preferred_lft forever
      inet6 fe80::f816:3eff:fea8:5429/64 scope link nodad
         valid_lft forever preferred_lft forever
  [root@dev0 ~]#

  [root@dev0 ~]# openstack port delete 555a9272-c9df-4a05-9f08-752c91c5a4c9
  [root@dev0 ~]# neutron l3-agent-list-hosting-router r1
  neutron CLI is deprecated and will be removed in the Z cycle. Use openstack CLI instead.
  +--------------------------------------+------+----------------+-------+----------+
  | id                                   | host | admin_state_up | alive | ha_state |
  +--------------------------------------+------+----------------+-------+----------+
  | 6fa92056-ca25-42e0-aee4-c4e744008239 | dev2 | True           | :-)   | active   |
  | 8fbda128-dc9c-4b3b-be1b-bb3f11ad1447 | dev1 | True           | :-)   | standby  |
  +--------------------------------------+------+----------------+-------+----------+
  [root@dev0 ~]#

  [root@dev0 ~]# ip netns exec snat-5d9d6fee-6652-4843-9f7c-54c11899d721 ip a s qg-d77e624d-87
  28: qg-d77e624d-87: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
      link/ether fa:16:3e:a8:54:29 brd ff:ff:ff:ff:ff:ff
      inet 10.136.17.172/20 scope global qg-d77e624d-87
         valid_lft forever preferred_lft forever
      inet6 fe80::f816:3eff:fea8:5429/64 scope link nodad
         valid_lft forever preferred_lft forever
  [root@dev0 ~]# ssh dev2 ip netns exec snat-5d9d6fee-6652-4843-9f7c-54c11899d721 ip a s qg-d77e624d-87
  28: qg-d77e624d-87: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
      link/ether fa:16:3e:a8:54:29 brd ff:ff:ff:ff:ff:ff
      inet 10.136.17.172/20 scope global qg-d77e624d-87
         valid_lft forever preferred_lft forever
      inet6 fe80::f816:3eff:fea8:5429/64 scope link nodad
         valid_lft forever preferred_lft forever
  [root@dev0 ~]#

  [root@dev0 ~]# arping -c 1 -I eth0 10.136.17.172
  ARPING 10.136.17.172 from 10.136.20.188 eth0
  Unicast reply from 10.136.17.172 [FA:16:3E:A8:54:29]  1.537ms
  Unicast reply from 10.136.17.172 [FA:16:3E:A8:54:29]  2.383ms
  Sent 1 probes (1 broadcast(s))
  Received 2 response(s)
  [root@dev0 ~]#

  
  As you can see, after deleting the HA port, we got a doubling of the ARP responses, which can lead to further problems in the roiting.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2008270/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next