yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1652909] [NEW] [vpnaas]UnicodeEncodeError occurs when using chinese charactors in vpn connection

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: siyingchun <1652909@xxxxxxxxxxxxxxxxxx>
Date: Wed, 28 Dec 2016 08:42:39 -0000
Reply-to: Bug 1652909 <1652909@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

In our Newton test environment, we found VPN connection can not be
created successfully when you  inputting Chinese characters which is our
national language as its name, or when you use Chinese characters as its
PSK, an unexpected and terrible result will happen，for instance, VPN
peers can access each other by using PSKs which include different
Chinese characters.

BTW, errors as below:
[vpnaas]UnicodeEncodeError: 'ascii' codec can't encode characters in position 20-21: ordinal not in range(128) occur when add IKE Policy with chinese charactors in vpn connection


22016-12-13 11:22:43.824 587926 WARNING neutron.agent.linux.iptables_manager [req-c888cc5c-0cc9-4070-85ad-514b0a552285 ebfa941f10994c859ad61ce074ea6f4a 69db65f43832456581518c876bd94706 - - -] Duplicate iptables rule detected. This may indicate a bug in the the iptables rule generation code. Line: -A neutron-vpn-agen-POSTROUTING -s 192.168.10.0/24 -d 192.168.2.0/24 -m policy --dir out --pol ipsec -j ACCEPT 
2016-12-13 11:22:43.825 587926 WARNING neutron.agent.linux.iptables_manager [req-c888cc5c-0cc9-4070-85ad-514b0a552285 ebfa941f10994c859ad61ce074ea6f4a 69db65f43832456581518c876bd94706 - - -] Duplicate iptables rule detected. This may indicate a bug in the the iptables rule generation code. Line: -A neutron-vpn-agen-POSTROUTING -s 192.168.10.0/24 -d 192.168.0.0/24 -m policy --dir out --pol ipsec -j ACCEPT 
2016-12-13 11:22:43.826 587926 WARNING neutron.agent.linux.iptables_manager [req-c888cc5c-0cc9-4070-85ad-514b0a552285 ebfa941f10994c859ad61ce074ea6f4a 69db65f43832456581518c876bd94706 - - -] Duplicate iptables rule detected. This may indicate a bug in the the iptables rule generation code. Line: -A neutron-vpn-agen-POSTROUTING -s 192.168.10.0/24 -d 192.168.2.0/24 -m policy --dir out --pol ipsec -j ACCEPT 
2016-12-13 11:22:43.826 587926 WARNING neutron.agent.linux.iptables_manager [req-c888cc5c-0cc9-4070-85ad-514b0a552285 ebfa941f10994c859ad61ce074ea6f4a 69db65f43832456581518c876bd94706 - - -] Duplicate iptables rule detected. This may indicate a bug in the the iptables rule generation code. Line: -A neutron-vpn-agen-POSTROUTING -s 192.168.10.0/24 -d 192.168.0.0/24 -m policy --dir out --pol ipsec -j ACCEPT 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher [req-c888cc5c-0cc9-4070-85ad-514b0a552285 ebfa941f10994c859ad61ce074ea6f4a 69db65f43832456581518c876bd94706 - - -] Exception during message handling: 'ascii' codec can't encode characters in position 20-21: ordinal not in range(128) 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last): 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher incoming.message)) 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185, in _dispatch 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 669, in vpnservice_updated 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher self.sync(context, [router] if router else []) 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs) 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 823, in sync 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher self._sync_vpn_processes(vpnservices, sync_router_ids) 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 847, in _sync_vpn_processes 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher process.update() 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 238, in update 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher self.enable() 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 256, in enable 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher self.ensure_configs() 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/libreswan_ipsec.py", line 61, in ensure_configs 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher super(LibreSwanProcess, self).ensure_configs() 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 353, in ensure_configs 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher self.vpnservice) 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 184, in ensure_config_file 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher utils.replace_file(config_file_name, config_str) 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/debtcollector/removals.py", line 242, in wrapper 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs) 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 177, in replace_file 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib64/python2.7/socket.py", line 316, in write 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher data = str(data) # XXX Should really reject non-string non-buffers 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher UnicodeEncodeError: 'ascii' codec can't encode characters in position 20-21: ordinal not in range(128) 
2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher 


* Precondition: 
   You have a large scale environment or a small test one which includes vpnaas.
  	
* Step-by-step:
   1. Go to horizon > switch project to Winters
   2. Create IKE policy with Chinese characters 
   3. Create IPSec policy, vpn services 
   4. create VPN connection with IKE policy(e.g. Chinese characters) 
   5. Check VPN status 
    
    
* Expect result: 
   vpn connection can be created successfully without errors 

* Actual result: 
   errors can be found in /var/log/neutron/vpn-agent.log 

* Version:
   Openstack Newton, deployed with Fuel 10.0
   Ubuntu Ubuntu 16.04.1 LTS, running kernel 4.4.0-57-generic
   Neutron version 5.1.0
   VPN 7.0.0

** Affects: neutron
     Importance: Undecided
     Assignee: siyingchun (wintersi)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => siyingchun (wintersi)

** Summary changed:

- [vpnaas]UnicodeEncodeError occurs when adding IKE Policy with chinese charactors in vpn connection
+ [vpnaas]UnicodeEncodeError occurs when using chinese charactors in vpn connection

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1652909

Title:
  [vpnaas]UnicodeEncodeError occurs when using chinese charactors in vpn
  connection

Status in neutron:
  New

Bug description:
  In our Newton test environment, we found VPN connection can not be
  created successfully when you  inputting Chinese characters which is
  our national language as its name, or when you use Chinese characters
  as its PSK, an unexpected and terrible result will happen，for
  instance, VPN peers can access each other by using PSKs which include
  different Chinese characters.

  BTW, errors as below:
  [vpnaas]UnicodeEncodeError: 'ascii' codec can't encode characters in position 20-21: ordinal not in range(128) occur when add IKE Policy with chinese charactors in vpn connection

  
  22016-12-13 11:22:43.824 587926 WARNING neutron.agent.linux.iptables_manager [req-c888cc5c-0cc9-4070-85ad-514b0a552285 ebfa941f10994c859ad61ce074ea6f4a 69db65f43832456581518c876bd94706 - - -] Duplicate iptables rule detected. This may indicate a bug in the the iptables rule generation code. Line: -A neutron-vpn-agen-POSTROUTING -s 192.168.10.0/24 -d 192.168.2.0/24 -m policy --dir out --pol ipsec -j ACCEPT 
  2016-12-13 11:22:43.825 587926 WARNING neutron.agent.linux.iptables_manager [req-c888cc5c-0cc9-4070-85ad-514b0a552285 ebfa941f10994c859ad61ce074ea6f4a 69db65f43832456581518c876bd94706 - - -] Duplicate iptables rule detected. This may indicate a bug in the the iptables rule generation code. Line: -A neutron-vpn-agen-POSTROUTING -s 192.168.10.0/24 -d 192.168.0.0/24 -m policy --dir out --pol ipsec -j ACCEPT 
  2016-12-13 11:22:43.826 587926 WARNING neutron.agent.linux.iptables_manager [req-c888cc5c-0cc9-4070-85ad-514b0a552285 ebfa941f10994c859ad61ce074ea6f4a 69db65f43832456581518c876bd94706 - - -] Duplicate iptables rule detected. This may indicate a bug in the the iptables rule generation code. Line: -A neutron-vpn-agen-POSTROUTING -s 192.168.10.0/24 -d 192.168.2.0/24 -m policy --dir out --pol ipsec -j ACCEPT 
  2016-12-13 11:22:43.826 587926 WARNING neutron.agent.linux.iptables_manager [req-c888cc5c-0cc9-4070-85ad-514b0a552285 ebfa941f10994c859ad61ce074ea6f4a 69db65f43832456581518c876bd94706 - - -] Duplicate iptables rule detected. This may indicate a bug in the the iptables rule generation code. Line: -A neutron-vpn-agen-POSTROUTING -s 192.168.10.0/24 -d 192.168.0.0/24 -m policy --dir out --pol ipsec -j ACCEPT 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher [req-c888cc5c-0cc9-4070-85ad-514b0a552285 ebfa941f10994c859ad61ce074ea6f4a 69db65f43832456581518c876bd94706 - - -] Exception during message handling: 'ascii' codec can't encode characters in position 20-21: ordinal not in range(128) 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last): 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher incoming.message)) 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185, in _dispatch 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 669, in vpnservice_updated 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher self.sync(context, [router] if router else []) 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs) 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 823, in sync 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher self._sync_vpn_processes(vpnservices, sync_router_ids) 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 847, in _sync_vpn_processes 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher process.update() 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 238, in update 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher self.enable() 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 256, in enable 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher self.ensure_configs() 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/libreswan_ipsec.py", line 61, in ensure_configs 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher super(LibreSwanProcess, self).ensure_configs() 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 353, in ensure_configs 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher self.vpnservice) 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron_vpnaas/services/vpn/device_drivers/ipsec.py", line 184, in ensure_config_file 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher utils.replace_file(config_file_name, config_str) 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/debtcollector/removals.py", line 242, in wrapper 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs) 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 177, in replace_file 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib64/python2.7/socket.py", line 316, in write 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher data = str(data) # XXX Should really reject non-string non-buffers 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher UnicodeEncodeError: 'ascii' codec can't encode characters in position 20-21: ordinal not in range(128) 
  2016-12-13 11:22:43.885 587926 ERROR oslo_messaging.rpc.dispatcher 

  
  * Precondition: 
     You have a large scale environment or a small test one which includes vpnaas.
    	
  * Step-by-step:
     1. Go to horizon > switch project to Winters
     2. Create IKE policy with Chinese characters 
     3. Create IPSec policy, vpn services 
     4. create VPN connection with IKE policy(e.g. Chinese characters) 
     5. Check VPN status 
      
      
  * Expect result: 
     vpn connection can be created successfully without errors 

  * Actual result: 
     errors can be found in /var/log/neutron/vpn-agent.log 

  * Version:
     Openstack Newton, deployed with Fuel 10.0
     Ubuntu Ubuntu 16.04.1 LTS, running kernel 4.4.0-57-generic
     Neutron version 5.1.0
     VPN 7.0.0

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1652909/+subscriptions
Follow ups

[Bug 1652909] Re: [vpnaas]UnicodeEncodeError occurs when using chinese charactors in vpn connection
From: OpenStack Infra, 2018-06-21