yahoo-eng-team team mailing list archive

[OpenStack Infra <1580377@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/355163
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=2acdcedb594d06e7c9f4f14e017c918a4a65c6b2
Submitter: Jenkins
Branch:    master

commit 2acdcedb594d06e7c9f4f14e017c918a4a65c6b2
Author: yujie <yujie@xxxxxxxxxxxxxxxxxxxx>
Date:   Mon Aug 15 22:14:01 2016 +0800

    Delete related conntrack when delete vm
    
    When deleting vm, the conntrack for this vm is still
    exist. This patch deletes removed port's conntrack entries
    only in port's hosting node. Another patch [1] removes
    this port's conntrack entries in remote hosts.
    
    [1] https://review.openstack.org/#/c/352440/
    
    Closes-Bug: #1580377
    Change-Id: Ia8132a62050eaa5e01377f653337b5792f158e2f


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1580377

Title:
  conntrack entry is not deleted when security_groups_member_updated

Status in neutron:
  Fix Released

Bug description:
  When remote group member changed, conntrack entry which should be
  deleted is not deleted.

  How to reproduce:
  * create a VM(VM1) on host-1 (net-a, default security-group) (ex. 10.0.0.3)
  * create a VM(VM2) on host-2 (net-a, default security-group) (ex. 10.0.0.4)
  * ssh from VM1(10.0.0.3) to VM2(10.0.0.4)
  ---
  host-2:$ sudo conntrack -L |grep 10.0
  tcp      6 431986 ESTABLISHED src=10.0.0.3 dst=10.0.0.4 sport=45074 dport=22 src=10.0.0.4 dst=10.0.0.3 sport=22 dport=45074 [ASSURED] mark=0 zone=1 use=1

  host-2:$ sudo ipset list
  Name: NIPv492469920-ef76-44af-98c7-
  Type: hash:net
  Revision: 4
  Header: family inet hashsize 1024 maxelem 65536
  Size in memory: 16824
  References: 1
  Members:
  10.0.0.4
  10.0.0.3
  ---

  * terminate VM1  (nova delete VM1) 
  expected: the conntrack entry showed above is deleted.
  actual: not deleted
  ---
  host-2:$ sudo conntrack -L |grep 10.0
  tcp      6 431986 ESTABLISHED src=10.0.0.3 dst=10.0.0.4 sport=45074 dport=22 src=10.0.0.4 dst=10.0.0.3 sport=22 dport=45074 [ASSURED] mark=0 zone=1 use=1

  host-2:$ sudo ipset list
  Name: NIPv492469920-ef76-44af-98c7-
  Type: hash:net
  Revision: 4
  Header: family inet hashsize 1024 maxelem 65536
  Size in memory: 16824
  References: 1
  Members:
  10.0.0.4
  ---

  Applied:
  liberty, mitaka, master

  Investigation:
  summary - devices_with_updated_sg_members is consumed by remove_devices_filter unintentionally.
  * when ovs-agent receives security_groups_member_updated RPC call, 
    sg_ids and affected devices are registered to self.firewall.devices_with_updated_sg_members.
    (original intention is that it is handled when refresh_firewall is called. but...)
  * in the main loop of ovs-agent process_deleted_ports is executed before process_network_ports.
    process_deleted_ports calls self.sg_agent.remove_devices_filter.
    if there is any deleted port, 
    remove_devices_filter
    -> defer_apply_off
    -> _remove_conntrack_entries_from_sg_update
    -> _clean_deleted_remote_sg_members_conntrack_entries
    is called. 
    at this time pre_sg_members and sg_members are same since no port info is 
    updated in remove_devices_filter. so no conntrack entry is removed. 
    but nonetheless devices_with_updated_sg_members is cleared !!
  * afterwards
    process_network_ports
    -> setup_port_filters -> refresh_firewall -> defer_apply_off
    ...-> _clean_deleted_remote_sg_members_conntrack_entries
    is called.
    at this time pre_sg_members and sg_members are different since port info was updated.
    but no conntrack entry is removed since devices_with_updated_sg_members was cleared.

  Note:
  deleting conntrack entries was introduced by https://bugs.launchpad.net/neutron/+bug/1335375.
  note that conntrack zone was per network at this time. (per port now)
  The fix of 1335375 is complicated and I wonder it is incomplete (ex. no care of egress & remote-group rule).
  How about simplify to just record affected ports and do "conntrack -D -w <port's zone id>" 
  for affected ports ?

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1580377/+subscriptions