openstack team mailing list archive

[Pete Zaitcev <zaitcev@xxxxxxxxxx>]

On Mon, 4 Jun 2012 14:40:21 +0900
iryoung jeong <iryoung@xxxxxxxxx> wrote:

> Anyway, current status is that I make your PatchSet 3 passed all
> unit/functests with webob 1.1 and only 4 tests failed with webob 1.2b3.
> After I compared my patch & your patchsets, I thought they looks similar,
> therefore I rebased my patch based on PatchSet 3 and keep trying to make
> them pass functests.

Here's a fix for the problem with proxy_logging middleware.
As I mentioned I run functional tests against a reasonably normal
server, not SAIO.

Please look at this and incorporate in change 8195. However, I must
note that I have 22 functional tests still failing on WebOb 1.2b3.
I'm going to continue working on resolving these issues.

-- Pete

commit f629016447e6156a8f9da7d94358427e47c2f375
Author: Pete Zaitcev <zaitcev@xxxxxxxxxxxxxxxxx>
Date:   Wed Jun 6 19:34:06 2012 -0600

    Update 1 for Iryoung's patch #3 for change 8195
    
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/swift/common/middleware/proxy_logging.py", line 205, in iter_response
        client_disconnect or input_proxy.client_disconnect)
      File "/usr/lib/python2.7/site-packages/swift/common/middleware/proxy_logging.py", line 120, in log_request
        the_request = quote(unquote(req.path))
      File "/usr/lib/python2.7/site-packages/webob/request.py", line 482, in path
        bpath = bytes_(self.path_info, self.url_encoding)
      File "/usr/lib/python2.7/site-packages/webob/descriptors.py", line 68, in fget
        return req.encget(key, encattr=encattr)
      File "/usr/lib/python2.7/site-packages/webob/request.py", line 174, in encget
        return val.decode(encoding)
      File "/usr/lib64/python2.7/encodings/utf_8.py", line 16, in decode
        return codecs.utf_8_decode(input, errors, True)
    UnicodeDecodeError: 'utf8' codec can't decode byte 0x8f in position 12: invalid start byte

diff --git a/swift/common/middleware/proxy_logging.py b/swift/common/middleware/proxy_logging.py
index 66c853c..55ab14d 100644
--- a/swift/common/middleware/proxy_logging.py
+++ b/swift/common/middleware/proxy_logging.py
@@ -44,6 +44,9 @@ from webob import Request
 
 from swift.common.utils import get_logger, get_remote_client, TRUE_VALUES
 
+import codecs
+utf8_decoder = codecs.getdecoder('utf-8')
+
 
 class InputProxy(object):
     """
@@ -115,7 +118,13 @@ class ProxyLoggingMiddleware(object):
         req = Request(env)
         if client_disconnect:  # log disconnected clients as '499' status code
             status_int = 499
-        the_request = quote(unquote(req.path))
+        # Do not try to access req.path: crash in req.encget if invalid UTF-8.
+        req_path = env.get('PATH_INFO','')
+        if isinstance(req_path, str):
+            (req_path,_len) = utf8_decoder(req_path,'replace')
+        # Fold back to UTF-8 or else quote() tracebacks immediately below.
+        req_path = req_path.encode('utf-8')
+        the_request = quote(unquote(req_path))
         if req.query_string:
             the_request = the_request + '?' + req.query_string
         logged_headers = None