yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1478604] [NEW] VPNaaS: openswan process isn't stopped at removing the router from l3 agent

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Hiroyuki Ito <ito.hiroyuki_01@xxxxxxxxxxxxxx>
Date: Mon, 27 Jul 2015 15:20:37 -0000
Reply-to: Bug 1478604 <1478604@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

When removing a router from l3 agent, the openswan process on its router isn't
stopped though the router's network namespace is deleted. I think the process 
should be stopped at least because it increases abandoned openswan processes.

Reproduce procedure:
--------------------
I found this problem at the following devstack environment:
stack@ubuntu-com1:~/devstack$ git show
commit 9cdde34319feffc7f1e27a4ffea43eae40eb6536

The operation I did is as follows:

1) Crete "IPsecSiteConnection" resource

The namespaces including the openswan process was as follows:
    root@ubuntu-com1:~# ip netns | grep 82174423-af6a-4c0d-b637-d34fa7a6b24b
    qrouter-82174423-af6a-4c0d-b637-d34fa7a6b24b
The openswan process on 82174423-af6a-4c0d-b637-d34fa7a6b24b was running like
   the following:
    root@ubuntu-com1:~# ps aux | grep ipsec/82174423-af6a-4c0d-b637-d34fa7a6b24b
    root 5183 0.0 0.0 94072 3992 ? Ss 18:46 0:00 /usr/lib/ipsec/pluto --ctlbase 
    /opt/stack/data/neutron/ipsec/82174423-af6a-4c0d-b637-d34fa7a6b24b/var/run/p
    luto --ipsecdir /opt/stack/data/neutron/ipsec/82174423-af6a-4c0d-b637-d34fa7
    a6b24b/etc --use-netkey --uniqueids --nat_traversal --secretsfile /opt/stack
    /data/neutron/ipsec/82174423-af6a-4c0d-b637-d34fa7a6b24b/etc/ipsec.secrets -
    -virtual_private %v4:172.16.200.0/24,%v4:172.16.100.0/24
    root 12553 0.0 0.0 11884 2204 pts/18 S+ 23:19 0:00 grep --color=auto ipsec/8
    2174423-af6a-4c0d-b637-d34fa7a6b24

2) Remove router which includes the 1)'s resource from the l3 agent

I removed 82174423-af6a-4c0d-b637-d34fa7a6b24b from the l3 agent by "neutron
   l3-agent-router-remove" cli.
   The namespaces on the node are as follows:
    stack@ubuntu-com1:~$ ip netns | grep 82174423-af6a-4c0d-b637-d34fa7a6b24b
    stack@ubuntu-com1:~$

3) Check processes on the node with 2)'s l3 agent

The openswan process was still running like the following:
    stack@ubuntu-com1:~$ ps aux | grep ipsec/82174423-af6a-4c0d-b637-d34fa7a6b24b
    root 5183 0.0 0.0 94072 3992 ? Ss 18:46 0:00 /usr/lib/ipsec/pluto --ctlbase 
    /opt/stack/data/neutron/ipsec/82174423-af6a-4c0d-b637-d34fa7a6b24b/var/run/p
    luto --ipsecdir /opt/stack/data/neutron/ipsec/82174423-af6a-4c0d-b637-d34fa7
    a6b24b/etc --use-netkey --uniqueids --nat_traversal --secretsfile /opt/stack
    /data/neutron/ipsec/82174423-af6a-4c0d-b637-d34fa7a6b24b/etc/ipsec.secrets -
    -virtual_private %v4:172.16.200.0/24,%v4:172.16.100.0/24
In the vpn agent log, the following error message was outputed:
    2015-07-27 23:20:57.415 ^[[00;32mDEBUG oslo_concurrency.lockutils Releasing 
    semaphore "iptables-qrouter-82174423-af6a-4c0d-b637-d34fa7a6b24b" from (pid=
    19216) lock /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutil
    s.py:210
    2015-07-27 23:20:57.415 ERROR neutron.callbacks.manager Error during notific
    ation for neutron_vpnaas.services.vpn.vpn_service.router_removed_actions rou
    ter, after_delete
    .....
    2015-07-27 23:20:57.415 TRACE neutron.callbacks.manager Command: ['ip', 'net
    ns', 'exec', u'qrouter-82174423-af6a-4c0d-b637-d34fa7a6b24b', 'iptables-save
    ', '-c']
    2015-07-27 23:20:57.415 TRACE neutron.callbacks.manager Exit code: 1
    2015-07-27 23:20:57.415 TRACE neutron.callbacks.manager Stdin:
    2015-07-27 23:20:57.415 TRACE neutron.callbacks.manager Stdout:
    2015-07-27 23:20:57.415 TRACE neutron.callbacks.manager Stderr: Cannot open 
    network namespace "qrouter-82174423-af6a-4c0d-b637-d34fa7a6b24b": No such fi
    le or directory

** Affects: neutron
     Importance: Undecided
     Assignee: Hiroyuki Ito (ito-hiroyuki-01)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Hiroyuki Ito (ito-hiroyuki-01)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1478604

Title:
  VPNaaS: openswan process isn't stopped at removing the router from l3
  agent

Status in neutron:
  New

Bug description:
  When removing a router from l3 agent, the openswan process on its router isn't
  stopped though the router's network namespace is deleted. I think the process 
  should be stopped at least because it increases abandoned openswan processes.

  Reproduce procedure:
  --------------------
  I found this problem at the following devstack environment:
  stack@ubuntu-com1:~/devstack$ git show
  commit 9cdde34319feffc7f1e27a4ffea43eae40eb6536

  The operation I did is as follows:

  1) Crete "IPsecSiteConnection" resource

  The namespaces including the openswan process was as follows:
      root@ubuntu-com1:~# ip netns | grep 82174423-af6a-4c0d-b637-d34fa7a6b24b
      qrouter-82174423-af6a-4c0d-b637-d34fa7a6b24b
  The openswan process on 82174423-af6a-4c0d-b637-d34fa7a6b24b was running like
     the following:
      root@ubuntu-com1:~# ps aux | grep ipsec/82174423-af6a-4c0d-b637-d34fa7a6b24b
      root 5183 0.0 0.0 94072 3992 ? Ss 18:46 0:00 /usr/lib/ipsec/pluto --ctlbase 
      /opt/stack/data/neutron/ipsec/82174423-af6a-4c0d-b637-d34fa7a6b24b/var/run/p
      luto --ipsecdir /opt/stack/data/neutron/ipsec/82174423-af6a-4c0d-b637-d34fa7
      a6b24b/etc --use-netkey --uniqueids --nat_traversal --secretsfile /opt/stack
      /data/neutron/ipsec/82174423-af6a-4c0d-b637-d34fa7a6b24b/etc/ipsec.secrets -
      -virtual_private %v4:172.16.200.0/24,%v4:172.16.100.0/24
      root 12553 0.0 0.0 11884 2204 pts/18 S+ 23:19 0:00 grep --color=auto ipsec/8
      2174423-af6a-4c0d-b637-d34fa7a6b24

  2) Remove router which includes the 1)'s resource from the l3 agent

  I removed 82174423-af6a-4c0d-b637-d34fa7a6b24b from the l3 agent by "neutron
     l3-agent-router-remove" cli.
     The namespaces on the node are as follows:
      stack@ubuntu-com1:~$ ip netns | grep 82174423-af6a-4c0d-b637-d34fa7a6b24b
      stack@ubuntu-com1:~$

  3) Check processes on the node with 2)'s l3 agent

  The openswan process was still running like the following:
      stack@ubuntu-com1:~$ ps aux | grep ipsec/82174423-af6a-4c0d-b637-d34fa7a6b24b
      root 5183 0.0 0.0 94072 3992 ? Ss 18:46 0:00 /usr/lib/ipsec/pluto --ctlbase 
      /opt/stack/data/neutron/ipsec/82174423-af6a-4c0d-b637-d34fa7a6b24b/var/run/p
      luto --ipsecdir /opt/stack/data/neutron/ipsec/82174423-af6a-4c0d-b637-d34fa7
      a6b24b/etc --use-netkey --uniqueids --nat_traversal --secretsfile /opt/stack
      /data/neutron/ipsec/82174423-af6a-4c0d-b637-d34fa7a6b24b/etc/ipsec.secrets -
      -virtual_private %v4:172.16.200.0/24,%v4:172.16.100.0/24
  In the vpn agent log, the following error message was outputed:
      2015-07-27 23:20:57.415 ^[[00;32mDEBUG oslo_concurrency.lockutils Releasing 
      semaphore "iptables-qrouter-82174423-af6a-4c0d-b637-d34fa7a6b24b" from (pid=
      19216) lock /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutil
      s.py:210
      2015-07-27 23:20:57.415 ERROR neutron.callbacks.manager Error during notific
      ation for neutron_vpnaas.services.vpn.vpn_service.router_removed_actions rou
      ter, after_delete
      .....
      2015-07-27 23:20:57.415 TRACE neutron.callbacks.manager Command: ['ip', 'net
      ns', 'exec', u'qrouter-82174423-af6a-4c0d-b637-d34fa7a6b24b', 'iptables-save
      ', '-c']
      2015-07-27 23:20:57.415 TRACE neutron.callbacks.manager Exit code: 1
      2015-07-27 23:20:57.415 TRACE neutron.callbacks.manager Stdin:
      2015-07-27 23:20:57.415 TRACE neutron.callbacks.manager Stdout:
      2015-07-27 23:20:57.415 TRACE neutron.callbacks.manager Stderr: Cannot open 
      network namespace "qrouter-82174423-af6a-4c0d-b637-d34fa7a6b24b": No such fi
      le or directory

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1478604/+subscriptions
Follow ups

[Bug 1478604] Re: VPNaaS: openswan process isn't stopped at removing the router from l3 agent
From: Armando Migliaccio, 2016-11-17