yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1586082] [NEW] vpnaas: "Failed to enable vpn process on router " due to wrong rundir

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Thomas Bechtold <tbechtold@xxxxxxxx>
Date: Thu, 26 May 2016 15:27:05 -0000
Reply-to: Bug 1586082 <1586082@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

When using vpnaas with strongswan 5.1 and strongswan uses as "piddir"
(see "ipsec --piddir) something different than "/var/run", the error is:

2016-05-26 15:22:22.541 29695 DEBUG neutron.agent.linux.utils [req-0a2127cd-125e-4d4b-b6db-04085baf5602 74cdd700184948c2b7fad2caa003ec2f a14c2b3f29d444db8a99176bac54b26b - - -] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-2691a9d2-fb5e-4d86-9023-ab3681bda8d3', 'neutron-vpn-netns-wrapper', '--mount_paths=/etc:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc,/var/run:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run', '--cmd=ipsec,up,e4c7ea00-db44-4387-9417-399e15ef410c'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
2016-05-26 15:22:22.899 29695 ERROR neutron.agent.linux.utils [req-0a2127cd-125e-4d4b-b6db-04085baf5602 74cdd700184948c2b7fad2caa003ec2f a14c2b3f29d444db8a99176bac54b26b - - -] 
Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-2691a9d2-fb5e-4d86-9023-ab3681bda8d3', 'neutron-vpn-netns-wrapper', '--mount_paths=/etc:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc,/var/run:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run', '--cmd=ipsec,up,e4c7ea00-db44-4387-9417-399e15ef410c']
Exit code: 7
Stdin: 
Stdout: Command: ['mount', '--bind', '/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc', '/etc'] Exit code: 0 Stdout:  Stderr: Command: ['mount', '--bind', '/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run', '/var/run'] Exit code: 0 Stdout:  Stderr: Command: ['ipsec', 'up', 'e4c7ea00-db44-4387-9417-399e15ef410c'] Exit code: 7 Stdout:  Stderr: 
Stderr: 2016-05-26 15:22:22.856 31074 INFO neutron.common.config [-] Logging enabled!
2016-05-26 15:22:22.856 31074 INFO neutron.common.config [-] /usr/bin/neutron-vpn-netns-wrapper version 7.0.5.dev91
2016-05-26 15:22:22.863 31074 INFO neutron_vpnaas.services.vpn.common.netns_wrapper [-] /var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc has been bind-mounted in /etc
2016-05-26 15:22:22.866 31074 INFO neutron_vpnaas.services.vpn.common.netns_wrapper [-] /var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run has been bind-mounted in /var/run

2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec [req-0a2127cd-125e-4d4b-b6db-04085baf5602 74cdd700184948c2b7fad2caa003ec2f a14c2b3f29d444db8a99176bac54b26b - - -] Failed to enable vpn process on router 2691a9d2-fb5e-4d86-9023-ab3681bda8d3
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec Traceback (most recent call last):
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 260, in enable
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec     self.start()
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/strongswan_ipsec.py", line 166, in start
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec     self._execute([self.binary, 'up', ipsec_site_conn['id']])
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/strongswan_ipsec.py", line 107, in _execute
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec     extra_ok_codes=extra_ok_codes)
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 898, in execute
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec     log_fail_as_error=log_fail_as_error, **kwargs)
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec     raise RuntimeError(m)
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec RuntimeError: 
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-2691a9d2-fb5e-4d86-9023-ab3681bda8d3', 'neutron-vpn-netns-wrapper', '--mount_paths=/etc:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc,/var/run:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run', '--cmd=ipsec,up,e4c7ea00-db44-4387-9417-399e15ef410c']
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec Exit code: 7
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec Stdin: 
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec Stdout: Command: ['mount', '--bind', '/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc', '/etc'] Exit code: 0 Stdout:  Stderr: Command: ['mount', '--bind', '/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run', '/var/run'] Exit code: 0 Stdout:  Stderr: Command: ['ipsec', 'up', 'e4c7ea00-db44-4387-9417-399e15ef410c'] Exit code: 7 Stdout:  Stderr: 
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec Stderr: 2016-05-26 15:22:22.856 31074 INFO neutron.common.config [-] Logging enabled!
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec 2016-05-26 15:22:22.856 31074 INFO neutron.common.config [-] /usr/bin/neutron-vpn-netns-wrapper version 7.0.5.dev91
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec 2016-05-26 15:22:22.863 31074 INFO neutron_vpnaas.services.vpn.common.netns_wrapper [-] /var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc has been bind-mounted in /etc
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec 2016-05-26 15:22:22.866 31074 INFO neutron_vpnaas.services.vpn.common.netns_wrapper [-] /var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run has been bind-mounted in /var/run
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec 
2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec 
2016-05-26 15:22:22.901 29695 DEBUG neutron.agent.linux.utils [req-0a2127cd-125e-4d4b-b6db-04085baf5602 74cdd700184948c2b7fad2caa003ec2f a14c2b3f29d444db8a99176bac54b26b - - -] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-2691a9d2-fb5e-4d86-9023-ab3681bda8d3', 'neutron-vpn-netns-wrapper', '--mount_paths=/etc:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc,/var/run:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run', '--cmd=ipsec,status'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
2016-05-26 15:22:23.248 29695 DEBUG neutron.agent.linux.utils [req-0a2127cd-125e-4d4b-b6db-04085baf5602 74cdd700184948c2b7fad2caa003ec2f a14c2b3f29d444db8a99176bac54b26b - - -] 


The piddir parameter can only be set during compile time so the neutron-vpn-agent must use the correct directory when bind mount var/run.

** Affects: neutron
     Importance: Undecided
     Assignee: Thomas Bechtold (toabctl)
         Status: In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1586082

Title:
  vpnaas: "Failed to enable vpn process on router " due to wrong rundir

Status in neutron:
  In Progress

Bug description:
  When using vpnaas with strongswan 5.1 and strongswan uses as "piddir"
  (see "ipsec --piddir) something different than "/var/run", the error
  is:

  2016-05-26 15:22:22.541 29695 DEBUG neutron.agent.linux.utils [req-0a2127cd-125e-4d4b-b6db-04085baf5602 74cdd700184948c2b7fad2caa003ec2f a14c2b3f29d444db8a99176bac54b26b - - -] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-2691a9d2-fb5e-4d86-9023-ab3681bda8d3', 'neutron-vpn-netns-wrapper', '--mount_paths=/etc:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc,/var/run:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run', '--cmd=ipsec,up,e4c7ea00-db44-4387-9417-399e15ef410c'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
  2016-05-26 15:22:22.899 29695 ERROR neutron.agent.linux.utils [req-0a2127cd-125e-4d4b-b6db-04085baf5602 74cdd700184948c2b7fad2caa003ec2f a14c2b3f29d444db8a99176bac54b26b - - -] 
  Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-2691a9d2-fb5e-4d86-9023-ab3681bda8d3', 'neutron-vpn-netns-wrapper', '--mount_paths=/etc:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc,/var/run:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run', '--cmd=ipsec,up,e4c7ea00-db44-4387-9417-399e15ef410c']
  Exit code: 7
  Stdin: 
  Stdout: Command: ['mount', '--bind', '/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc', '/etc'] Exit code: 0 Stdout:  Stderr: Command: ['mount', '--bind', '/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run', '/var/run'] Exit code: 0 Stdout:  Stderr: Command: ['ipsec', 'up', 'e4c7ea00-db44-4387-9417-399e15ef410c'] Exit code: 7 Stdout:  Stderr: 
  Stderr: 2016-05-26 15:22:22.856 31074 INFO neutron.common.config [-] Logging enabled!
  2016-05-26 15:22:22.856 31074 INFO neutron.common.config [-] /usr/bin/neutron-vpn-netns-wrapper version 7.0.5.dev91
  2016-05-26 15:22:22.863 31074 INFO neutron_vpnaas.services.vpn.common.netns_wrapper [-] /var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc has been bind-mounted in /etc
  2016-05-26 15:22:22.866 31074 INFO neutron_vpnaas.services.vpn.common.netns_wrapper [-] /var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run has been bind-mounted in /var/run

  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec [req-0a2127cd-125e-4d4b-b6db-04085baf5602 74cdd700184948c2b7fad2caa003ec2f a14c2b3f29d444db8a99176bac54b26b - - -] Failed to enable vpn process on router 2691a9d2-fb5e-4d86-9023-ab3681bda8d3
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec Traceback (most recent call last):
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 260, in enable
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec     self.start()
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/strongswan_ipsec.py", line 166, in start
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec     self._execute([self.binary, 'up', ipsec_site_conn['id']])
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/strongswan_ipsec.py", line 107, in _execute
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec     extra_ok_codes=extra_ok_codes)
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 898, in execute
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec     log_fail_as_error=log_fail_as_error, **kwargs)
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 159, in execute
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec     raise RuntimeError(m)
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec RuntimeError: 
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-2691a9d2-fb5e-4d86-9023-ab3681bda8d3', 'neutron-vpn-netns-wrapper', '--mount_paths=/etc:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc,/var/run:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run', '--cmd=ipsec,up,e4c7ea00-db44-4387-9417-399e15ef410c']
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec Exit code: 7
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec Stdin: 
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec Stdout: Command: ['mount', '--bind', '/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc', '/etc'] Exit code: 0 Stdout:  Stderr: Command: ['mount', '--bind', '/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run', '/var/run'] Exit code: 0 Stdout:  Stderr: Command: ['ipsec', 'up', 'e4c7ea00-db44-4387-9417-399e15ef410c'] Exit code: 7 Stdout:  Stderr: 
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec Stderr: 2016-05-26 15:22:22.856 31074 INFO neutron.common.config [-] Logging enabled!
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec 2016-05-26 15:22:22.856 31074 INFO neutron.common.config [-] /usr/bin/neutron-vpn-netns-wrapper version 7.0.5.dev91
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec 2016-05-26 15:22:22.863 31074 INFO neutron_vpnaas.services.vpn.common.netns_wrapper [-] /var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc has been bind-mounted in /etc
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec 2016-05-26 15:22:22.866 31074 INFO neutron_vpnaas.services.vpn.common.netns_wrapper [-] /var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run has been bind-mounted in /var/run
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec 
  2016-05-26 15:22:22.900 29695 ERROR neutron_vpnaas.services.vpn.device_drivers.ipsec 
  2016-05-26 15:22:22.901 29695 DEBUG neutron.agent.linux.utils [req-0a2127cd-125e-4d4b-b6db-04085baf5602 74cdd700184948c2b7fad2caa003ec2f a14c2b3f29d444db8a99176bac54b26b - - -] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-2691a9d2-fb5e-4d86-9023-ab3681bda8d3', 'neutron-vpn-netns-wrapper', '--mount_paths=/etc:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/etc,/var/run:/var/lib/neutron/ipsec/2691a9d2-fb5e-4d86-9023-ab3681bda8d3/var/run', '--cmd=ipsec,status'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
  2016-05-26 15:22:23.248 29695 DEBUG neutron.agent.linux.utils [req-0a2127cd-125e-4d4b-b6db-04085baf5602 74cdd700184948c2b7fad2caa003ec2f a14c2b3f29d444db8a99176bac54b26b - - -] 

  
  The piddir parameter can only be set during compile time so the neutron-vpn-agent must use the correct directory when bind mount var/run.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1586082/+subscriptions
Follow ups

[Bug 1586082] Re: vpnaas: "Failed to enable vpn process on router " due to wrong rundir
From: OpenStack Infra, 2016-06-01
[Bug 1586082] Re: vpnaas: "Failed to enable vpn process on router " due to wrong rundir
From: Thomas Bechtold, 2016-05-27