← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1818805] Re: Conntrack rules in the qrouter are not deleted when a fip is removed with dvr

 

Reviewed:  https://review.openstack.org/641747
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=0c6d76f210331c1f45ebb24c7d92a80044a15057
Submitter: Zuul
Branch:    master

commit 0c6d76f210331c1f45ebb24c7d92a80044a15057
Author: Rodolfo Alonso Hernandez <ralonsoh@xxxxxxxxxx>
Date:   Thu Mar 7 18:17:04 2019 +0000

    Remove conntrack rule when FIP is deleted
    
    Change-Id: Ia2bc8a7b5b27a5d4fbbfb7813cde7f022934bd22
    Closes-Bug: #1818805


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1818805

Title:
  Conntrack rules in the qrouter are not deleted when a fip is removed
  with dvr

Status in neutron:
  Fix Released

Bug description:
  If a fip ip is removed of a network with a distributed router:

  openstack server remove floating ip  X

  
  The conntrack rules aren't deleted in the qrouter and the qrouter continues doing nating of the ongoing connections.


  
  overcloud) [stack@undercloud-0 ~]$ openstack router show router
  +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  | Field                   | Value                                                                                                                                                                                                                                                                    |
  +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  | admin_state_up          | UP                                                                                                                                                                                                                                                                       |
  | availability_zone_hints |                                                                                                                                                                                                                                                                          |
  | availability_zones      | nova                                                                                                                                                                                                                                                                     |
  | created_at              | 2019-02-20T15:46:53Z                                                                                                                                                                                                                                                     |
  | description             |                                                                                                                                                                                                                                                                          |
  | distributed             | True                                                                                                                                                                                                                                                                     |
  | external_gateway_info   | {"network_id": "15a5c01e-4e42-4890-a850-db4f97bb5834", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "c59ae813-1df7-4a14-9eba-be2e35afa13e", "ip_address": "10.0.0.214"}]}                                                                                   |
  | flavor_id               | None                                                                                                                                                                                                                                                                     |
  | ha                      | False                                                                                                                                                                                                                                                                    |
  | id                      | d01c89b0-c2df-46e2-9c12-8d14b1c8ce9a                                                                                                                                                                                                                                     |
  | interfaces_info         | [{"subnet_id": "5e8ddfa7-d546-4f59-94d1-e2b65e8ecdb6", "ip_address": "10.2.0.8", "port_id": "06c6e9d3-2c6b-40b8-8919-92be6efd0153"}, {"subnet_id": "5e8ddfa7-d546-4f59-94d1-e2b65e8ecdb6", "ip_address": "10.2.0.1", "port_id": "c47b0417-7dbe-4434-8c50-72a78e6335a1"}] |
  | name                    | router                                                                                                                                                                                                                                                                   |
  | project_id              | 9447276fedbf4c4eab15494f8d187d97                                                                                                                                                                                                                                         |
  | revision_number         | 13                                                                                                                                                                                                                                                                       |
  | routes                  |                                                                                                                                                                                                                                                                          |
  | status                  | ACTIVE                                                                                                                                                                                                                                                                   |
  | tags                    |                                                                                                                                                                                                                                                                          |
  | updated_at              | 2019-03-05T11:31:34Z                                                                                                                                                                                                                                                     |
  +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  (overcloud) [stack@undercloud-0 ~]$ 
  (overcloud) [stack@undercloud-0 ~]$ 
  (overcloud) [stack@undercloud-0 ~]$ 
  (overcloud) [stack@undercloud-0 ~]$ openstack server list 
  +--------------------------------------+------------------------+--------+------------------------------------+--------+--------+
  | ID                                   | Name                   | Status | Networks                           | Image  | Flavor |
  +--------------------------------------+------------------------+--------+------------------------------------+--------+--------+
  | 8aad2992-b068-4378-83f8-965b59d04d8d | selfservice2-instance4 | ACTIVE | selfservice2=10.2.0.12, 10.0.0.210 | cirros | cirros |
  | 028e0696-e666-4c09-802a-49a126a6346d | selfservice2-instance3 | ACTIVE | selfservice2=10.2.0.27             | cirros | cirros |
  | 46432868-4d11-4c9d-a910-ddf246c78378 | selfservice2-instance2 | ACTIVE | selfservice2=10.2.0.14             | cirros | cirros |
  | fca6541e-d846-45fd-8970-8ba27ff708d6 | selfservice2-instance1 | ACTIVE | selfservice2=10.2.0.30             | cirros | cirros |
  +--------------------------------------+------------------------+--------+------------------------------------+--------+--------+

  $ ip a
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
      inet 127.0.0.1/8 scope host lo
         valid_lft forever preferred_lft forever
      inet6 ::1/128 scope host 
         valid_lft forever preferred_lft forever
  2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast qlen 1000
      link/ether fa:16:3e:6e:0e:f7 brd ff:ff:ff:ff:ff:ff
      inet 10.2.0.12/24 brd 10.2.0.255 scope global eth0
         valid_lft forever preferred_lft forever
      inet6 fe80::f816:3eff:fe6e:ef7/64 scope link 
         valid_lft forever preferred_lft forever
  $ hostname 
  selfservice2-instance4
  $ 

  $ 
  $ ping 8.8.8.8
  PING 8.8.8.8 (8.8.8.8): 56 data bytes
  64 bytes from 8.8.8.8: seq=0 ttl=116 time=49.554 ms
  64 bytes from 8.8.8.8: seq=1 ttl=116 time=48.893 ms
  64 bytes from 8.8.8.8: seq=2 ttl=116 time=48.897 ms
  64 bytes from 8.8.8.8: seq=3 ttl=116 time=48.988 ms
  ....

  [heat-admin@compute-1 ~]$ sudo su 
  [root@compute-1 heat-admin]# yum install conntrack-tools

  ....

  [root@compute-1 heat-admin]# conntrack -L
  icmp     1 29 src=10.2.0.12 dst=8.8.8.8 type=8 code=0 id=45825 src=8.8.8.8 dst=10.0.0.210 type=0 code=0 id=45825 mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
  conntrack v1.4.4 (conntrack-tools): 1 flow entries have been shown.
  [root@compute-1 heat-admin]# 


  (overcloud) [stack@undercloud-0 ~]$ openstack server remove floating ip selfservice2-instance4 10.0.0.210
  (overcloud) [stack@undercloud-0 ~]$ date 
  Wed Mar  6 04:59:40 EST 2019
  (overcloud) [stack@undercloud-0 ~]$ 

  Then connectivity is lost:

  64 bytes from 8.8.8.8: seq=813 ttl=116 time=48.753 ms
  64 bytes from 8.8.8.8: seq=814 ttl=116 time=48.895 ms


  
  because conntrack flow is not deleted:

  
  [root@compute-1 heat-admin]# conntrack -L
  icmp     1 29 src=10.2.0.12 dst=8.8.8.8 type=8 code=0 id=45825 src=8.8.8.8 dst=10.0.0.210 type=0 code=0 id=45825 mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
  conntrack v1.4.4 (conntrack-tools): 1 flow entries have been shown.

  [root@compute-1 heat-admin]# date
  Wed Mar  6 10:02:04 UTC 2019
  [root@compute-1 heat-admin]# conntrack -L
  icmp     1 29 src=10.2.0.12 dst=8.8.8.8 type=8 code=0 id=45825 src=8.8.8.8 dst=10.0.0.210 type=0 code=0 id=45825 mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
  conntrack v1.4.4 (conntrack-tools): 1 flow entries have been shown.
  [root@compute-1 heat-admin]# 


  If I delete the flow the connectiviy is recovered(same behavior that
  without dvr):


  [root@compute-1 heat-admin]# conntrack -D
  icmp     1 29 src=10.2.0.12 dst=8.8.8.8 type=8 code=0 id=45825 src=8.8.8.8 dst=10.0.0.210 type=0 code=0 id=45825 mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
  conntrack v1.4.4 (conntrack-tools): 1 flow entries have been deleted.
  [root@compute-1 heat-admin]# conntrack -L
  icmp     1 28 src=10.2.0.12 dst=8.8.8.8 type=8 code=0 id=45825 [UNREPLIED] src=8.8.8.8 dst=10.2.0.12 type=0 code=0 id=45825 mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
  conntrack v1.4.4 (conntrack-tools): 1 flow entries have been shown.
  [root@compute-1 heat-admin]# 

  ....

  64 bytes from 8.8.8.8: seq=812 ttl=116 time=48.791 ms
  64 bytes from 8.8.8.8: seq=813 ttl=116 time=48.753 ms
  64 bytes from 8.8.8.8: seq=814 ttl=116 time=48.895 ms



  64 bytes from 8.8.8.8: seq=980 ttl=117 time=49.979 ms
  64 bytes from 8.8.8.8: seq=981 ttl=117 time=49.164 ms
  64 bytes from 8.8.8.8: seq=982 ttl=117 time=49.524 ms
  64 bytes from 8.8.8.8: seq=983 ttl=117 time=49.143 ms

  
  ....

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1818805/+subscriptions


References