yahoo-eng-team team mailing list archive

[Kristian Kostecky <kris@xxxxxxxxxxx>]

Public bug reported:

In Keystone (2012.2.1) and future versions of keystone the s3_token
middleware will throw an exception when swift accepts a PUT of an object
that contains utf-8 characters. The exception is as follows from the
swift proxy-server:

'ascii' codec can't decode byte 0xc3 in position 42: ordinal not in
range(128):

#012Traceback (most recent call last):

#012  File "/usr/lib/python2.6/site-
packages/swift3-1.0.0-py2.6.egg/swift3/middleware.py", line 825, in
__call__#012 return self.handle_request(env, start_response)#012

File "/usr/lib/python2.6/site-
packages/swift3-1.0.0-py2.6.egg/swift3/middleware.py", line 895, in
handle_request#012    res = getattr(controller, req.method)(env,
start_response)#012

File "/usr/lib/python2.6/site-
packages/swift3-1.0.0-py2.6.egg/swift3/middleware.py", line 763, in
PUT#012    body_iter = self._app_call(env)#012

File "/usr/lib/python2.6/site-
packages/swift-1.7.5-py2.6.egg/swift/common/wsgi.py", line 265, in
_app_call#012    resp = iter(self.app(env, self._start_response))#012

File "/usr/lib/python2.6/site-
packages/swift-1.7.5-py2.6.egg/swift/common/middleware/tempurl.py", line
229, in __call__#012 return self.app(env, start_response)#012

File "/usr/lib/python2.6/site-
packages/keystone-2012.2.1-py2.6.egg/keystone/middleware/s3_token.py",
line 218, in __call__#012    new_tenant_name)#012UnicodeDecodeError:
'ascii' codec can't decode byte 0xc3 in position 42: ordinal not in
range(128)

The fix is simple, just change:

        environ['PATH_INFO'] = environ['PATH_INFO'].replace(account,
                                                            new_tenant_name)

        environ['PATH_INFO'] = environ['PATH_INFO'].decode('utf-8').replace(account,
                                                            new_tenant_name).encode('utf-8')


There may be a more complete way of making this utf-8 compliant, but this serves its purpose. There is a bug submitted to the swift3 project that I believe links to this issue: https://github.com/fujita/swift3/issues/65

** Affects: keystone
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Keystone.
https://bugs.launchpad.net/bugs/1284779

Title:
  Keystone s3_token middleware does not handle utf-8 encoded characters
  in PATHS from swift

Status in OpenStack Identity (Keystone):
  New

Bug description:
  In Keystone (2012.2.1) and future versions of keystone the s3_token
  middleware will throw an exception when swift accepts a PUT of an
  object that contains utf-8 characters. The exception is as follows
  from the swift proxy-server:

  'ascii' codec can't decode byte 0xc3 in position 42: ordinal not in
  range(128):

  #012Traceback (most recent call last):

  #012  File "/usr/lib/python2.6/site-
  packages/swift3-1.0.0-py2.6.egg/swift3/middleware.py", line 825, in
  __call__#012 return self.handle_request(env, start_response)#012

  File "/usr/lib/python2.6/site-
  packages/swift3-1.0.0-py2.6.egg/swift3/middleware.py", line 895, in
  handle_request#012    res = getattr(controller, req.method)(env,
  start_response)#012

  File "/usr/lib/python2.6/site-
  packages/swift3-1.0.0-py2.6.egg/swift3/middleware.py", line 763, in
  PUT#012    body_iter = self._app_call(env)#012

  File "/usr/lib/python2.6/site-
  packages/swift-1.7.5-py2.6.egg/swift/common/wsgi.py", line 265, in
  _app_call#012    resp = iter(self.app(env, self._start_response))#012

  File "/usr/lib/python2.6/site-
  packages/swift-1.7.5-py2.6.egg/swift/common/middleware/tempurl.py",
  line 229, in __call__#012 return self.app(env, start_response)#012

  File "/usr/lib/python2.6/site-
  packages/keystone-2012.2.1-py2.6.egg/keystone/middleware/s3_token.py",
  line 218, in __call__#012    new_tenant_name)#012UnicodeDecodeError:
  'ascii' codec can't decode byte 0xc3 in position 42: ordinal not in
  range(128)

  The fix is simple, just change:

          environ['PATH_INFO'] = environ['PATH_INFO'].replace(account,
                                                              new_tenant_name)

          environ['PATH_INFO'] = environ['PATH_INFO'].decode('utf-8').replace(account,
                                                              new_tenant_name).encode('utf-8')

  
  There may be a more complete way of making this utf-8 compliant, but this serves its purpose. There is a bug submitted to the swift3 project that I believe links to this issue: https://github.com/fujita/swift3/issues/65

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1284779/+subscriptions